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Abstract 


In this thesis, we study the optimal tradeoff of average delay, average service cost, and average 
utility for single server queueing models, with and without admission control. The continuous time 
and discrete time queueing models that we consider are motivated by cross-layer models for noisy 
point-to-point links, with random packet arrivals. We study the above tradeoff problem for a class 
of admissible policies, which are monotone and stationary. 

The solutions that we obtain for the above tradeoff problem are asymptotic in nature. For example, 
suppose we are interested in minimizing the average delay of packets, subject to a constraint on 
the average service cost of serving the packets. It is intuitive that to keep the queue stable the 
time average service rate of packets has to equal the time average arrival rate of packets. This in 
turn implies that queue stability requires a positive minimum average service cost expenditure. We 
obtain asymptotic bounds on the minimum average delay in the asymptotic regime K where the 
average service cost constraint is a small positive V more than the above minimum average service 
cost required for queue stability. We note that such asymptotic bounds can be used to obtain a first 
order characterization of the tradeoff curve, and are useful in identifying good families of scheduling 
policies, such as buffer partitioning policies. 

In this thesis, we obtain asymptotic lower bounds on the minimum average delay in the regime 3ft, for 
the cases for which lower bounds were previously not known, for admissible policies. The asymptotic 
characterization of the minimum average delay for admissible policies, for both continuous time and 
discrete time models, is obtained via new geometric bounds on the stationary probability distribution 
of the queue length, in the regime 3ft. The restriction to admissible policies, also enables us to obtain 
an intuitive explanation for the behaviour of the asymptotic lower bounds, using the above geometric 
bounds on the stationary probability distribution of the queue length. We observe that the shape 
of the stationary probability distribution, in the regime 3ft, determines the form of the asymptotic 
behaviour. 



It is common practice to approximate a queueing model, where the queue length evolution is on 
the non-negative integers, with a queueing model where the queue length evolution is on the non¬ 
negative real numbers and the service cost function being strictly convex, for analytical tractability. 
We compare the asymptotic bounds which are obtained for the approximate real valued queue 
evolution model with that of the original integer valued queue evolution model. We observe that 
for some cases the average delay does not grow to infinity, in the regime 3?, although the real valued 
approximate queueing model, with a strictly convex cost function, suggests that the average delay 
should grow without bound in the regime 3?. In other cases where the average delay does grow to 
infinity in the regime 3i?, our results illustrate that the approximate model strictly underestimates 
the behaviour of the tradeoff for the original model unless the service cost function is modelled as 
the piecewise linear lower convex envelope of the service cost function for the original integer valued 
queueing model. 

The geometric bounds on the stationary probability distribution of the queue length also lead to 
asymptotic bounds on any optimal admissible policy, in the regime 3ft. The asymptotic order bounds 
are independent of the exact service cost function, and are not available in previous work. For buffer 
partitioning policies, the bounds also show how buffer partitions have to scale with V. 

We then apply the above asymptotic lower bounds to the motivating applications, discussed above. 
We develop geometric bounds on the stationary probability distribution for admissible policies to 
analyse the tradeoff problem in other scenarios, such as: (i)queueing models for N-user single 
hop communication networks, (ii)queueing models with non-convex service cost functions, and 
(iii)queueing models with general holding costs. 
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CHAPTER 1 


Introduction 


In this thesis, we study the optimal tradeoff of average delay, average service cost, and average 
utility for single server queueing models, with and without admission control. The continuous 
time and discrete time queueing models that we consider are motivated by cross-layer models for 
noisy point-to-point links. The features that we model are: (i)random packet arrivals, (ii)control 
of service and/or arrival rates, (iii)packet service costs and/or utility, and (iv)fading at slow and 
fast time scales. Our objective is to characterize the minimum average delay of the packets, under 
an upper bound constraint on the average service cost and/or a lower bound constraint on the 
average throughput for systems with admission control. We are also motivated by the problem 
of characterizing the minimum average delay of randomly arriving message symbols which are 
transmitted over a noisy point-to-point link with no admission control, under an upper bound 
constraint on the average error rate of the message symbols. Such tradeoff problems arise in the 
study of cross layer scheduling algorithms for wireless communication networks |45] or in the study 
of processor speed scaling |T7]. 

In this thesis, we consider the performance of scheduling algorithms which optimally trade off 
average delay with other performance measures, as in [7], [45j, and [25J. Related problems include 
the design of cross layer scheduling algorithms for: (a) stabilizing a communication network, as in 
[Bj, [74j, [4lJ, [25j, jl5j, [38J, [63J, [66J, [64j, and [651], or (b) minimizing a delay measure, as in 
tzgj, t40J, and m. 

We study the above tradeoff problem for a class of monotone policies, which we call admissible 
policies. Monotone policies are stationary policies, i.e., the service rate (and the number of packets 
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admitted, for queues with admission control) at a time is a functiorj^ only of the current state of 
the queu^ rather than a function of the whole history of evolution as well as the current state of 
the queue. For monotone policies, the expected service rate for a queue length is non-decreasing 
as a function of the queue length. Intuitively, if the average delay is to be minimized subject to 
a constraint on the average service cost, for deterministic stationary policies, as the queue length 
increases, the service rate should also increase. We note that intuitively although the service rate 
should increase, the amount by which the service rate increases depends on the corresponding 
increase in the service cost. So in practice, monotone policies are usually used. Furthermore, in 
certain cases it can be shown that the optimal policy for the above tradeoff problem is in fact 
monoton^ Motivated by this reason, as well as the above intuition, we consider the tradeoff 
problem for the class of admissible policies only. The class of admissible policies is a subset of the 
class of monotone policies, possessing some additional properties, which makes their analysis more 
amenable. 

We consider several variations of the above tradeoff problem. The solutions to the tradeoff problems 
that we address in this thesis are of the following form. Consider an example of an infinite buffer 
queueing model with service rate control, but no admission control. We are interested in minimizing 
the average delay subject to a constraint on the average service cost. It is intuitive that to keep the 
queue stable the time average service rate of packets has to equal the time average arrival rate of 
packets. This in turn implies that queue stability requires a positive minimum average service cost 
expenditure. We obtain asymptotic bounds on the minimum average delay in the asymptotic regime 

where the average service cost constraint is a small positive V more than the above minimum 
average service cost required for queue stability. We note that such asymptotic bounds can be used 
to obtain a first order characterization of the tradeoff curve. Furthermore, such bounds can also be 
used to identify good families of scheduling policies, as in |7|. 

We note that asymptotic upper bounds (see [45J) as well as asymptotic lower bounds (see j7] 
and [43j) on the minimum average delay in the regime 5? are available for a variety of queueing 
models. Flowever, asymptotic lower bounds are not known in many cases. We obtain asymptotic 
lower bounds for the minimum average delay in the regime 5R, for these cases, for admissible policies. 
Additionally, we also obtain asymptotic bounds on the structure of admissible policies which achieve 
the above asymptotic lower bounds. The method by which we derive these bounds, which is different 
from previous approaches, also leads to geometric bounds on the stationary probability of the queue 
length, in the regime 5R. These bounds have the added advantage of directly providing intuition for 
the behaviour of asymptotic lower bounds in the regime 5R. 

Using the above asymptotic lower bounds on the minimum average delay and the already available 

^this function could be randomized. 

^e.g., the current queue length or the current queue length and an auxiliary state variable such as the fade state. 

^The monotonicity property is obtained using a Markov decision theoretic formulation of the tradeoff problem as 
in CQ], [25], or [7] 
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asymptotic upper bounds we obtain a complete asymptotic characterization of the tradeoff between 
average delay and average service cost for several single server queueing models, in the regime 5R, 
for admissible policies. We start with an informal introduction to the queueing models and the 
tradeoff problems considered in this thesis. 


1.1 Introduction to the queueing models and the tradeoff problem 

We note that the primary features of the point-to-point links are: (i)random packet arrivals, 
(ii)control of service and/or arrival rates, (iii)packet service costs and/or utility, and (iv)fading 
at slow and fast time scales. These features are captured by two single server controlled queueing 
models in this thesis. Initial insights into the asymptotic behaviour of the tradeoff are obtained 
by studying a continuous time queueing model with exponential interarrival times and exponential 
service requirements, in Chapters 2 and 3. These insights are used in characterizing the asymptotic 
behaviour of the tradeoff for discrete time models in the following chapters. 

For the purposes of this introductory discussion, a queue is defined to be stable (or more precisely, 
mean rate stable as in |45i|) under a policy if the time average service rate equals the time average 
arrival rate, although in later chapters we use stronger notions of stability. 

In the asymptotic regime 5R, asymptotic upper and lower bounds on the average delay can be 
obtained from asymptotic upper and lower bounds on the average queue length by applying Little’s 
law with lower and upper bounds on the average throughput respectively. Therefore, we focus on 
the average queue length instead of average delay throughout this thesis. 

1.1.1 A continuous time state dependent M/M/1 model 

The state dependent M/M/1 queueing model is a birth death process with state being the queue 
length, and with state dependent birth (or arrival) rates and death (or service) rates, as shown in 
the transition diagram in Figure [T7l| We note that the control policy 7 is the choice of the arrival 
rates {X{q),q G Z+) and the service rates {p{q),q G \ { 0 }), as a function of the queue length 
q, from sets Xx and X^ respectively. 

The average queue length Q{'y) for a particular policy 7 is the time average of the expectation of 
the queue length Q{t), where Q{t) is the state of the birth death process at time t under 7 . We 
assume that utility is accrued at the rate of u{X{Q{t))) and service cost is incurred at the rate 
of c{p{Q{t))), where u{X) is a non-decreasing concave function of A and c{p) is a non-decreasing 
convex function of p. For the policy 7 , the average utility rate 1 /( 7 ) and average service cost rate 
0 ( 7 ) are defined as the time averages of the expectation of the utility rate u{X{Q{t))) and the 
expectation of the service cost rate c{p{Q{t))) respectively. The general tradeoff problem that we 
consider is the minimization of ( 5 ( 7 ), subject to a lower bound constraint Uc on 1 /( 7 ), and an upper 
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A(0) A(l) A(2) 


Hq) A(q'+1) 



Figure 1.1: Transition diagram for the continuous birth-death queueing model 


bound constraint Cc on C{'y), over all policies 7 , i.e., 

minimize 

7 

such that [ 7 ( 7 ) > Uc and C{'y) < Cc- (1.1) 

In the following, problems in the above constrained optimization form are called constrained tradeoff 
problems. We note that one way to analyse the above constrained tradeoff problem is to consider 
its unconstrained Lagrange dual, the dual function of which is as follows: 

minimize Q{-f) + j3i{C{-i) - Cc) - - Uc), (1.2) 

7 

where /3i and (32 are non-negative Lagrange multipliers. In the following, such unconstrained dual 
problems are called unconstrained tradeoff problems. 

We primarily consider the case where X{q) is fixed to be a A G TLa such that u{X) > Uc, so that the 
tradeoff problem reduces to 


minimize <^( 7 ), 

7 

such that C{'y) < Cc- (1.3) 

We note that as in the case of (O' we have the following unconstrained dual function: 

minimize < 5 ( 7 ) -F /3i(C(7) - Cc), (1.4) 

7 

where /3i > 0. 

The optimal values of ( |1.1D and ( [Q , as a function of their respective constraints, are referred to 
as the tradeoff curve in the following discussion. We note that in the context of communication 
networks, the function u{.) is usually assumed to be linear, so that 1 /( 7 ) is the average throughput. 


4 




But for other applications, such as those in |5j, u{.) could be a strictly concave function. 


The optimization problem ( |1.2p and its variants (such as ( |1.4[ )) have been formulated as Markov 
decision problems (MDP) and analyzed by many authors, e.g. [5j, [jOjl, and [25|- They show 


that there exists a monotone optimal policy 7*(/3i,/32) for (1.2) (7*(/3i,/32) is such that X{q) is a 
non-increasing function and is a non-decreasing function of q). Also, from [35], we have that 
if Cc = C(7*(/3 i,/32 )) and Uc = f7(7*(/3i,/32)), then 7*(/3i,/32) is also optimal for dl.lj ). Therefore, 
for at least such values of Cc and Uc there exist monotone optimal policies for ( |11D . This motivates 
us to consider ( |11D for a class of admissible policies, which are monotone. 

The asymptotic regime 5R : We consider the tradeoff problems ( |11D and (1.3) in the asymptotic 
regime 3? where the average service cost constraint Cc is arbitrarily close to the minimum average 
service cost required for stability. It turns out that the minimum average service cost required for 
stability is c{u~^{uc)) (where the inverse u~^{.) of n(.) is assumed to exist) for ( |1.1[ ) and c(A) for 
Q. Therefore, the asymptotic regimes 3? for problems ( |11D and ( |1-3D are defined as the regime 
in which Cc i c{u~^{uc)) and Cc i c(A) respectively. 


1.1.2 A discrete time queueing model 


In Chapters 4, 5, and 6, we consider discrete time single server queueing models with random batch 


arrivals and batch service. We now introduce a general form of this model, shown in Figure 1.2 


In each slot m > 1, we assume that a random number R[m] of packets arrives into the system. 



Figure 1.2: The discrete time single server queueing model with a single queue 


with an arrival rate of A per slot. In the slot, A[m] < R[m] arrivals are admitted into the 
queue, which is assumed to have infinite buffer space. We assume that there is a random process 
{H[m\) which models the environment in which the queue is evolving, e.g., this could be the fade 
state for a point-to-point wireless link, which stays constant for the duration of a slot. We assume 
that H[m] is known at the start of every slot m. The queue length, at the slot boundaries, evolves 
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according to the evolution equation: 


Q[m] = Q[m — 1] — S'fm] + A[m],m > 1, 

where (^[O] = qo, 5[m] < m.m.{SmaxjQ[m — 1]), and A[m] < R[m\. We note that S[m\ and A[m] 
are the control variables. The service batch size 5[m] in slot m is assumed to be chosen as a 
randomized function of the history 

a[m] = {go, H[l], S[l], R[1],Q[1], H[2], 5[2], R[2],Q[2],... ,Q[m - 2],H[m - 1]), 

and the current queue length Q[m — 1] and fade state H[m]. while the arrival batch size A[m] in 
slot m is assumed to be chosen as a randomized function of the history a[m], Q[m — 1], H[m], and 
the current number of arrivals R[m]. The choice of the sequence ((5[m], A[m]),m > 1) constitutes 
the policy 7 for this discrete time model. 

The average queue length <5(7) for a policy 7 is defined to be the time average of the expectation 
of the queue length Q[m]. We assume that service cost is incurred at the rate of P{H[m], S'[m]) in 
slot m, where P{h,s) is a service cost function which is assumed to be non-decreasing and convex 
in s for every h. We note that the cost function P{h,s) models the cost incurred in transmission 
of packets, e.g., P{h,s) could be the expected number of packets that are received in error when 
a batch of s packets are jointly encoded and transmitted when the environment state is h, or 
P{h,s) could be the power expended in transmission of s packets when the fade state is h. The 
average service cost P{'y) for 7 is defined as the time average of the expectation of the service cost 
P{H[m\, 

We define the average throughput ^(7) for 7 as the time average of the expectation of the admitted 
arrival batch size yl[m]. The performance measure that we are interested in is the utility of the 
average throughput, u{A{'^)), achieved by 7, where u{.) is a non-decreasing and concave utility 
function. The general tradeoff problem that we consider is the minimization of <5(7) subject to a 
lower bound constraint Uc on u{A{'^)) and an upper bound constraint Pc on P{^) over all policies 
7, i.e.. 


minimize Q{'y), 

7 

such that u{A{'^)) > Uc and ^*( 7 ) < Pc- (1-5) 

As for the state dependent M/M/1 model, we have the following unconstrained dual function: 

minimize Qi-y) +/3i {P{'y) - Pc) - (32 {u{A{'y)) - Uc) , (1.6) 

where (3i and (32 are non-negative. 
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As for the state dependent M/M/1 model, we primarily consider the case where A[m] = R[m], i.e., 
with no admission control. Then the tradeoff problem is: 

minimize Qij), 

7 

such that Pil) < Pc, (1-7) 

where we have assumed that Uc is such that A > u~^{uc) (where the inverse of n(.) is 

assumed to exist), so that for stable policies the utility constraint is satisfied. Similar to ( |1.6D , we 
have the unconstrained dual function: 


minimize Q{j) + /3i {P{j) - Pc) , 


( 1 . 8 ) 


where /3i > 0. The optimal values of ( |1.5[ ) and ( |1.7[ ) as a function of their respective constraints 
are referred to as the tradeoff curve in the following discussion. 


As for the state dependent M/M/1 model, for the discrete time model, again using a MDP for¬ 
mulation, it is possible to show that (e.g. [7|, |25j, and jlj) there exists a stationary monotone 
optimal policy for ( |1.6D for each pair of /3i and (32- Again, if Uc and Pc are respectively equal to 
the utility and average service cost for the above optimal policy (for some /3i and j52), then this 
monotone policy is also optimal (see [35]) for ( |1.5D . This motivates us to consider ( |1.5D for a class 
of admissible policies, which are monotone. 


The asymptotic regime JR : We consider the tradeoff problems ( |1.5D and the asymptotic 

regime JR where the average service cost constraint Pc is arbitrarily close to the minimum average 
service cost required for stability. It turns out that this minimum average service cost required for 
stability is a function of Uc and A for ( |1.5D and ( |1.7[ ) respectively. Because of the similarities in its 
properties with those of c(.) for the state dependent M/M/1 model, the minimum average service 
cost required for stability is denoted as c(.) for ( |1.5[ ) and ( |1.7D also. We note that if there is only 
one environment state, say /iq, then c(s) = P{ho,s). The asymptotic regime JR for problems (1.5) 
and (|1.7D is defined as the regime in which Pc i c{u~^{uc)) and Pc i c(A) respectively. 


A multiqueue model : We also consider a multiqueue single server queueing model, with N 
queues being served by a single server as shown in Figure [T73| The model is a straightforward 
generalization of the single queue model discussed above. We assume that there is an environment 
variable 77„[m] associated with the r^th 

queue. The vector of N environment variables is denoted as 
H[m\ = ..., The vector of N queue lengths at the start of slot m is denoted as 

Q[m — 1] = {Qi[m — 1],..., (5Ar[m — 1]). We note that in this case the batch service vector. S']??!], 
is a vector function of the history cj[m] for the N queues, the current queue length vector Q[m — 1], 
and the current environment state H[m] as in the single queue case. The batch of arrivals which 
are admitted, A[m], is a vector function of the history a[m] for the N queues, Q[m — 1], H[m], 
and the vector of current arrivals R[m]. 
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The service cost is a scalar function P{h, s) of the service vector s and the environment vector h. 
For a particular policy 7, we are interested in the total average queue length (5(7), which is the sum 
of the average queue lengths for the N queues. The average service cost is the time average of the 
expectation of P{H[m], S[m]). We also assume that there are individual lower bound constraints 
on the u{An{'y)) for each queue. Other definitions are straightforward extensions of the definitions 
for the single queue model. The asymptotic regime 3? for the multiqueue model is similar to that 



Figure 1.3: The discrete time single server queueing model with N queues 

for the single queue case. We consider the problem of minimizing the average queue length in 
the asymptotic regime 3?, where the scalar cost constraint approaches the minimum average power 
required for mean rate stability. 


1.2 Literature survey 


Tradeoff problems for both continuous time and discrete time single server queueing models has 
been addressed by many researchers. To review what is known for such tradeoff problems, let us 
consider the problem (|1.3D. One of the first questions that can be asked is that of feasibility; for 


what values of Cc are there feasible solutions to (1.3)? Such questions are commonly answered 


using Lyapunov drift arguments as in, [45j and t4l|. The next question that can be asked is that 
of the existence of an optimal policy for ( |1.3D . Such questions are commonly answered by posing 

or |2j. Under 


the problem (1.3) as a constrained Markov decision problem (CMDP) as in 
certain technical assumptions, the existence of an optimal policy which is also stationary can be 
shown using results as in [2j, [28j, or [27]. In certain cases, it may be possible to show that there 
is a stationary deterministic optimal policy for (|1.3[|p] In such cases, it is also possible to convert 


the CMDP (1.3) into an unconstrained Markov decision problem (MDP) using suitable Lagrange 


^Stationary deterministic optimal policies are such that at a time, the service rate (and the arrival rate for models 
with admission control) is chosen as a deterministic function of the system state. 
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multipliers as in 0. For the state dependent M/M/1 model, the approach is then to find an 
optimal policy for the above MDP. We note that the optimal policy for (1.3) specifies the optimal 
service rate and arrival rate X*{q) as a function of queue length, to operate the system in 

order to minimize the time average of the single stage cost rate. Characterization of the optimal 
policy using the MDP approach yields structural properties, which are useful in reducing the search 
space for the optimal policy, for example see |7Dj, |25j, [3|, and j5|. Surveys of the above approach 
can be found in [321 and 


We first review the results which are available for the continuous time model. In most cases, a 
monotonicity property of the optimal policy for ( |1.2D is obtained, i.e., is a non-decreasing 

function of q and X*{q) is a non-increasing function of q. Stidham and Weber |7Dj show that fL*{q) 
is non-decreasing and X*{q) is non-increasing, for a state dependent M/G/1 model, where the 
objective is to minimize the expected total cost from any initial queue length by serving customers 
until the queue length is zero, which is related to the average cost criterion. George and Harrison 
[25] show that fJ,*{q) is non-decreasing in q, for a state dependent M/M/1 model with a Poisson 
arrival process of rate 1, for an MDP of the form in 0. Similar results have also been obtained 
by Ata in [3i|, and Ata and Shneorson in [5j. This monotonicity property of ^J,*{q) and X*{q) is 
used to motivate the definition of ar/m/ss/b/^ policies in Chapters 2 and 3, which are policies with 
non-decreasing service rates fi{q) and non-increasing arrival rates X{q) as a function of q. We 
then obtain an asymptotic characterization of the tradeoff problems ( |1.1[ ) and (1.3) for the class 
of admissible policies. We note that whenever the solution of ( |1.1[ ) coincides with that of its dual 
(obtained by optimizing (1.2) over /3i and (32 > 0), then the optimal policy for dl.lj) is admissible. 


For the discrete time model, from a CMDP formulation for ( |1.7[ ), it can be shown that there exists 
an optimal policy which chooses the service batch size S[m] as a function S*{Q[m — l],H[m]) of 
the current queue length and environment state, if the arrival process {R[m]) and the environment 
process {H[m]) are IID. The optimal rate or batch size S*{q, h) which has to be served as a function 
of the current queue length q and environment state h, so as to minimize the average queue length for 
a given constraint on the average service cost, can be characterized. Berry and Gallager [7], Collins 
and Cruz [18], Agarwal et al. [1], and Goyal et.al. |29| consider a fading point to point link with no 
admission control, where they study the tradeoff problem ( |1.7D . They use a Lagrangian relaxation 
of the CMDP as in ( |1.8[ ) to obtain that in many cases, S*{q,h) is a deterministic function s*{q,h) 
which is monotonically non-decreasing in q for every h. The monotonicity property of s*{q,h) is 
used in motivating the definition of admissible policies in Chapters 4, 5, and 6. We obtain an 
asymptotic characterization of the tradeoff problems ( |1.5D and ( |1.7D for the class of admissible 
policies. We note that whenever the solution of dl.Sj) coincides with that of its dual (obtained by 


^Admissible policies are monotone policies. Since admissible policies are stationary, the queue evolution process 
under admissible policies is a Markov process. Admissible policies are monotone policies which are such that the 
Markov queue evolution process possesses nice properties such as aperiodicity, irreducibility, positive recurrence, and 
finite mean queue length. 
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optimizing ( | 1 . 6 D over /?i and ^2 > 0 ), then the optimal policy for ( |1.5D is admissible. A similar 
observation holds for the solution of the tradeoff problem ( |1.7[ ). 

Going beyond the above monotonicity property of s*{q, h), Berry and Gallager [7] also characterized 
the asymptotic order behaviour of the tradeoff curve in the regime of large average delay. It was 
shown that when the average delay is allowed to be large, the average transmitter power can be 
made arbitrarily close to c(A). It was also shown that if the power P{h, s) expended in transmission 
of s packets is a strictly convex function of s for fixed fade state h, then the average queue length 
is ri if the average transmitter power constraint is V more than c(A) as F | 0 (this is known 

as the Berry-Gallager lower bound). This asymptotic characterization of the average transmitter 
power was motivated by the asymptotic characterization of the average distortion of an information 
source obtained by Tse [76j. 

We note that the order behaviour provides a first order characterization of the tradeoff curve. Fur¬ 
thermore, the order characterization provides a criterion to identify a good famil^ of transmission 
policies. The authors in |7] suggest that the family of buffer partitioning policies achieves the 
order bound but were unable to prove this. Buffer partitioning policies partition the buffer into 
two regions, and use an average batch service rate less than A in the lower region and an average 
batch service rate greater than A in the higher region to drive the average queue length towards the 
partitioning value. 


Neely |43j extended the Berry-Gallager lower bound to single hop networks and presented a back¬ 
pressure based (TOGA) algorithm which achieves the lower bound to within a logarithmic factor, 
i.e., the algorithm achieves an average delay of 0 ^;^log(^)^ when the average transmitter 
power is at most V more than c(A). Neely [43] also observed that if the transmitter power P{h,s) 
is a piecewise linear function of s for every h, then the above rate of increase of the average delay 
can be improved. We note that if P{h,s) is piecewise linear in s,V/i, then the function c(A) is 
a piecewise linear function of A. If (A,c(A)) lies on a linear portion of the function c(.), it was 
observed [53] that there is a family of policies for which the average delay is O (log(y)) if the 
average transmitter power is V more than c(A). It was also observed |43j that for all values of A 
and P{h, s) convex in s,V/i, there is a family of policies for which the average delay is O (■^) if the 
average transmitter power is V more than c(A). 


In [44], Neely considered the problem (1.5), with u{x) = x and P{h,s) a strictly convex function 
in s for every h. He observed that if the transmitter is allowed to drop a non-zero fraction of the 
customers arriving into the queue, such that ^( 7 ) > pX,0 < p < I, then the average queue length 
grows only as O (log (^)) rather than O log (^)^ if the average transmitter power is V more 
than c(A). Extensions to more general networks and other formulations can be found in [4 


family of transmission policies is a set of policies with common structure, e.g., a set of parametrized policies 
that do not serve below a threshold (parameter) while serving a particular batch size (another parameter) above the 
threshold. 
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We note that an order optimality result was obtained by Ramaiyan [50J for a particular birth death 
queueing model, for the average queue length at the relay node for a two-way relay link using 
network coding, in the regime 3?, for monotone policies. Ramaiyan et al. {51] also obtained the 
optimal tradeoff of average queueing delay and average transit delay for a two-hop vehicular relay 
network. Asymptotic upper bounds on the minimum average delay for general wireless networks 
using network coding, under an average power constraint, was obtained in [KJ. 


An asymptotic upper bound for the tradeoff curve corresponding to ( |1.7D has been obtained for 
the case where {A[m\) and {H[m]) are ergodic Markov processes in [45l Section 4.9] and |3Dj. In 
[45l Theorem 4.12] and [30|, it has been shown that if {A[m]) and {H[m]) are ergodic Markov 
processes, then for a sequence of Quadratic Lyapunov Algorithm (QLA) policies, parametrized by a 
sequence V the average queue length is O (^) for an average power V more than c(A). Order 
optimality has also been explored for finite buffer systems. In |6l Chapter 6] it is shown that for a 
finite buffer discrete time queueing model, as the buffer size B goes to infinity, for any sequence 
of policies such that the buffer overflow probability is o(^), the average service cost is at least 
Q (^) more than c(A). 


We note that, for the tradeoff problems (1.5) and ( |1.7D , although asymptotic upper bounds on the 
minimum average queue length are known, asymptotic lower bounds are not available in many cases. 
We note that such asymptotic lower bounds are significant, since they may help in determining the 
best possible tradeoff. Let V be the difference between the average service cost constraint Pc and 
the minimum average service cost for stability, c(A), in the asymptotic regime 3?, where 17 0. The 

known asymptotic lower bounds on the minimum average queue length, along with the details of 
the models analysed, and the asymptotic upper bounds on the minimum average queue length are 
summarized in Table fTTl 


We note that several asymptotic lower bounds in Table [TT] have been derived under the assumption 
that the queue length and service batch size take values in ]R_|_ and the service cost function is 
strictly convex. In certain cases, these real valued queueing models are used as approximate models 
for queueing models where the queue length and the service batch size take values in Z+. We note 
that there are also scenarios, where modelling the queue length evolution to be on M_|_ is natural, 
such as when the queue is assumed to buffer a certain amount of error exponent as in {?]. 

An approximate solution to the tradeoff problem has been obtained by Ata et al. ]4] by approximating 
{Q[m]) by a diffusion process, which enables them to find the optimal policy for the control of 
the approximating diffusion process in closed form. The complete characterization of the optimal 
admission control policy for a continuous time queueing model was obtained in {48j. 


Bettesh and Shamai fl0| obtain approximations for s*{q,h) for every h, in the regime of large q, by 
solving the average cost optimality equation associated with the MDP ( |1.8[ ). For the MDP (1.8), 
Chen et al. |[T7j obtain approximations for s*{q) from a fluid approximation. However, we note that 
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Model details 

Asymptotic upper 

bound (Regime 3?) 

Asymptotic lower 

bound (Regime 3f?) 

1 

Berry-Gallager power delay tradeoff [7j; 
P{h,s) strictly convex in s,V/i 



2 

Multiuser Berry-Gallager power delay 
tradeoff [43|; P{h,s) strictly convex in 
s,V/i 


^(w) 

3 

Multiuser Berry-Gallager power delay 
tradeoff [43]; piecewise linear c(.), A is 
such that c(A) is on a piecewise linear 
portion of c(.) 

0 (log(f)) 

n(log(^)) shown for 
a specific example, not 

known in general 

4 

Multiuser Berry-Gallager power delay 
tradeoff [43]; piecewise linear c(.), A is 
any abscissa at which the slope of c(.) 
changes 

o(A 

Not known 

5 

Power delay tradeoff with lower bound 
constraint on average throughput [44J 

0 (log(f)) 

n(log(^)) but with 
single fade state 

6 

Utility delay tradeoff |42l 

O (log (7)) 

^^(log(f)) 

7 

Power delay tradeoff with Markov arrival 
and fading process [3Dj 


Not known 


Table 1.1: Some of the available asymptotic bounds on the minimum average queue length; except for case 
7 all other models assume that the arrival process and the fade process are IID, and except for cases 5 and 
6 all models have A[m] = R[m]. Also, all lower bounds are derived under the assumption that the queue 
length can take real values. 


the bounds on s*{q) depend on the form of the service cost functions. 

Motivated by the above survey of known results, we ask and try to answer the following questions 
in this thesis: 


1. We note that, in Table 1.1 there are several cases in which asymptotic lower bounds are not 
known. What are these asymptotic lower bounds? 

2. From Table |l7l| we observe that asymptotic lower bounds have the form of log (^) or 
What is the intuition behind such a behaviour? 


3. As stated before, for certain cases, asymptotic lower bounds have been derived for an approx¬ 
imate queueing model, where the queue evolution is real valued. Are the asymptotic lower or 
upper bounds different for the original integer valued queue evolution model? 

4. Bounds on s*(g,/ iq) have been obtained, which are dependent on the service cost function. 
However, the asymptotic order bounds on the minimum average queue length are dependent 
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only on certain properties of the service cost function, rather than its exact form. Can we 
obtain asymptotic order bounds on the policy which are independent of the exact form of the 
service cost function? 

In the next section, we briefly survey how these issues have been addressed in this thesis. 


1.3 Overview of the thesis and contributions 


This thesis consists of two parts. In the first part, comprising Chapters 2, 3, and 4, we consider 
the tradeoff problems ( |l.ip , ( |1.3[ ), ( |1.5[ ), and ( |1.7[ ) in their respective asymptotic regimes JR, for 
the class of admissible policies. The second part, comprising Chapters 5 and 6, primarily illustrates 
the application of the results obtained in the first part to the motivating resource tradeoff problems 
arising in point-to-point communication links. 


In this introductory discussion, the results are stated informally. We note that the results hold under 
further technical assumptions, which are stated in the respective chapters. 


We study the continuous time state dependent M/M/1 queueing model in Chapters 2 and 3. The 
main results which are obtained in Chapters 2 and 3 are summarized in Table [T^ For the first two 
cases, let V be the difference between Cc and c(A) in the asymptotic regime JR, where y 0. For 
the third case V is the difference between u(|u) and Uc, while for the fourth case V is the difference 
between Cc and c(u~^(uc))- We note that the first two cases are instances of ( |1.3[ ) while the fourth 
case is an instance of ( |1.1[ ). The third case is similar to (1.3), except that the roles of A(q) and 
/u(q) are interchanged, i.e., there is admission control with a fixed service rate. To the best of 
our knowledge, such asymptotic results for the tradeoff for the continuous time state dependent 
M/M/1 model, which may be of independent interest, are new. Thus, we obtain answers for the 
first question that we posed, for the continuous time queueing models. 


The insights obtained from these two chapters are then used in deriving asymptotic lower bounds 
for a discrete time model in Chapter 4. The correspondence between discrete time models and the 
continuous time models can be achieved by the choice of and the form of the functions 

c(.) and u(.). The motivation behind the choice of TA, and the form of the functions c(.) and 
u(.) is explained in Chapter 3. 

From the analysis in Chapters 2 and 3, we obtain the following intuition for the behaviour of 
the asymptotic lower bounds, which partly answers the second question that we posed. We note 
that for the state dependent M/M/1 model, for admissible policies, it can be shown that the 
stationary probability distribution of the queue length exists. The intuition for the behaviour of 
the asymptotic lower bounds is based on the shape of this stationary probability distribution in the 
asymptotic regime JR. We discuss the behaviour of the stationary probability distribution only for 
the cases where the minimum average queue length increases to infinity in the regime JR. 


13 



Control 

Service cost and utility functions 

Results (in the regime 5R, 
for admissible policies) 

Xfj, is discrete, X\ = {A} 

c(.) is piecewise linear 

Depending on A, minimum 

average queue length either 

increases to a finite value, is 
©(log(^)), or0(^) 

is a finite interval, Xx = 

{A} 

c(.) is strictly convex 

c(.) is piecewise linear 

Minimum average queue 

length is C! 

Depending on A, minimum 

average queue length either 

increases to a finite value, is 
©(log (17)). or0(iA) 

Xfj, = {p}, Xx is a finite in¬ 
terval 

u{.) is strictly concave 

u{.) is piecewise linear 

Minimum average queue 

length is C! 

Depending on A, minimum 

average queue length is 

^(log(F)). 

Xfj, is a finite interval, Xx is a 

finite interval 

c(.) is strictly convex, u{.) is 
strictly concave 

Minimum average queue 
length is 0 (log (^)) 


Table 1.2: List of asymptotic results derived for continuous time queueing models in Chapters 2 and 3. 


The behaviour of the stationary probability distribution of the queue length is determined by the 
behaviour of the stationary probability distribution of the service rates, which is in turn decided by 
the nature of the function c(^) at /i = A or u~^{uc) as the case may be. We consider non-idling 
admissible policies for the purpose of discussion. We find that in the regime 5R, the stationary 
probability of using a service rate of zero, goes to zero as 0{V). Then, intuitively, since the 
probability of this queue being empty goes to zero, the stationary probability distribution shifts to 
the right as shown in Figure [O] Therefore, the average queue length has to increase. We note 
that this intuition has been used in the design of tradeoff optimal policies in 

We note that for monotone admissible policies, the stationary probability distribution has the fol¬ 
lowing shape (as in Figure [O] ). The stationary probability distribution is monotonically increasing, 
then may or may not be constant for a set of queue lengths, and then is monotonically decreasing. 

In the asymptotic regime 5R, the probability of using certain service rates decreases to zero as 0{V), 
while the probability of using certain service rates increases. Consider the set of queue lengths, Qh, 
which are such that the stationary probability of using the service rates p{q),q G Qh does not 
decrease to zero. The different behaviours for the minimum average queue length, depends on (i) 
the shape of the stationary probability distribution for the set Qh, and (ii) the stationary probability 
of the smallest queue length in Qh- 
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Stationary probability distribution (7r(g)) 



Figure 1.4: Intuition for the behaviour of the stationary probability distribution in the regime 5ft 


Stationary probability distribution (7r(q)) 



Figure 1.5; Possibilities for the behaviour of the stationary probability distribution in the regime 5ft 

In the asymptotic regime 5R, the shape of the stationary probability distribution for the set of 
queue lengths Qh, can be either (SI) monotonically increasing, constant, and then monotonically 
decreasing, or (S2) constant. We shall see that this is decided by the extent of freedom that we 
have in the choice of X{q) and In the asymptotic regime 5ft, the stationary probability of the 

smallest queue length in Qh is either (PI) 0{V) or (P2) 0{\/V). We shall see that this is decided 
by the form of the function c(.) at /r = A or u~^{uc) as the case may be. The various possibilities 
are illustrated in Figure [T3| 

Then, the log (y) behaviour for the minimum average queue length arises because the stationary 
probability distribution of the queue length is (SI) and the stationary probability of the smallest 
queue length in Qh is (PI). The asymptotic y behaviour arises with (S2) and (PI). The asymptotic 
behaviour arises with (S2) and (P2). Although we discuss this in more detail in Chapters 2 and 
3, here we provide an example to illustrate the derivation of the log (y) behaviour. For a particular 
policy, we obtain a geometric upper bound of the form 7r(0)p'^ (p > 1) on the stationary probability 
7r(q) of the queue length. In the asymptotic regime 5ft we also show that 7r(0) = 0(V). Then 
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applying Markov inequality we obtain that the average queue length is a constant times log (^). 

We note that these results are obtained using geometric upper bounds on the stationary probability 
of the queue length. Under the assumptions that we have made regarding the form of c(.) and 
u{.), we therefore observe that the possible forms for asymptotic lower bounds are log (y), y, or 
By providing this intuition, we have successfully answered the second question, for admissible 
policies, for the continuous time model. 


Using bounds on the stationary probability distribution of queue length, we also obtain asymptotic 
bounds on the structure of order-optimal admissible policie^ in the regime JR. Since admissible 
policies are monotone, we note that corresponding to a set of service rates of S, there is a contiguous 
set or interval, Qs, of queue lengths such that p{q) e S,'dq e Qs- In both Chapter 2 and Chapter 
3, we obtain asymptotic bounds on the cardinality of Qs- These bounds are independent of the 
exact form of the functions c(.) and u(.). Thus, we obtain some answers for the fourth question 
that we have posed, for continuous time queueing models. 


In Chapter 2, we also apply the analysis to a flow-level resource allocation model for a wireless 
downlink and obtain asymptotic bounds on the tradeoff of average power and average number of 
flows. 


In Chapter 4, we consider the problem ( |1.7D , with a single environment state (ho), in the asymptotic 
regime 5R. We note that in this case the service cost c(s) = P(ho, s) is a function of the batch size s 
only. For admissible policies, it turns out that the stationary distribution of the queue length exists. 
Using the insights about the shape of the stationary distribution of the queue length in the regime 
3? from Chapters 2 and 3, we obtain two upper bounds on the stationary probability distribution of 
the queue length, one of which is an extension of the bound on stationary probability distribution 
obtained in Bertsimas et al. [;9j to the case where the service rate is dependent on the queue length. 
The bounds can be used to obtain the same intuition, as explained earlier for the continuous time 
model, for the discrete time model. The bounds are used to obtain the results that are summarized 


in Table 1.3 Thus, we answer the first and second questions that we have posed, for admissible 
policies. The bounds on the stationary probability distribution are also used to obtain asymptotic 
bounds on the cardinality of Qs, leading to some answers for the fourth question that we have 
posed. 


We note that our asymptotic results apply to the tradeoff problems in Table |l.l] under the assumption 
that the optimal policy lies in the class of admissible policies, which is true for many cases 

We also show that approximating the original integer valued queueing model by a real valued model 
with a strictly convex function, leads to the average queue length and average service cost being 
underestimated, in certain cases. We also analyse a case, which have not been hitherto identified. 


7 

8 


1.2 


these are admissible policies which achieve the asymptotic lower bounds in Table 
For example, whenever the solution to o coincides with that of its dual (obtained from ||1.8[|) 
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Model details 

Service cost function 

Results (in the regime 3?, 
for admissible policies) 

Q[m] evolves on Z+ 

P{ho,s) is piecewise linear in 

s 

Depending on A, minimum 

average queue length either 

increases to a finite value, is 
0 (log(^)), or0(^) 

Q[m] evolves on Z_|_; A[m] is 
ergodic 

P{ho,s) is piecewise linear in 

s 

Depending on A, minimum 

average queue length either 

increases to a finite value or 
is fl(log(^)) 

Q[m] evolves on IR+ 

P{ho, s) is strictly convex in s 

Minimum average queue 
length is Q (previously 

known [7] but re-derived here 
using our method). 


Table 1.3: List of asymptotic results derived for tradeoff problem (1.7) in Chapter 4 with single environment 
state /iQ- 


where the average queue length increases to only a finite value in the asymptotic regime 5R, for the 
original integer valued queueing model. Thus, we obtain some answers to the third question that 
we have posed. We also show that a more appropriate real valued approximation is one in which 
the service cost function is approximated as the piecewise linear lower envelope of the service cost 
function for the original integer valued queueing model. 


In Chapter 5, we consider problems (1.7) and (1.5), in the context of a point-to-point link with 
fast fading. In this chapter, the environment variable h models the fade state and P{h,s) is the 
power expended in transmitting s packets in fade state h. Let V be the difference between the 
power constraint Pc and c(A) (or Pc and c{u~^{uc))) in the asymptotic regime 3?, where F | 0. 
Using the results in Chapter 4, we obtain an asymptotic characterization of the tradeoff. The main 


results obtained in this chapter are summarized in Table 1.4 We also comment on the extension 


of these asymptotic results to: (a) a N user single hop network model (as in Figure 1.3) with 
Qnlm] assumed to evolve on Z+, (b) a model with admission control and ergodic arrival and fading 
processes, and (c) a model with no service cost, but for which we are interested in the tradeoff of 
utility and delay as in |42]. We note that except for the case where the minimum average queue 
length increases to only a finite value (which has been hitherto not identified in literature) we are 


able to obtain asymptotic lower bounds for all of the models in Table 1.1 for admissible policies. 


We consider the tradeoff of average delay with average error rate for a point-to-point link in Chapter 
6 . The transmitter is assumed to use fixed or variable-length block coding. In this chapter, we 
interpret a packet as an information message symbol, which could be a bit. For fixed length block 
coding, we assume that s message symbols are encoded into a codeword of length Nc channel uses. 


17 














Model details 

Service cost function 

Results (in the regime 3?, 
for admissible policies) 

Q[m] evolves on Z+, A[m] = 
R[m], as in |43j 

P{h,s) is piecewise linear in 

s, V/r 

Depending on the arrival rate 

A, minimum average queue 

length either increases to 

only a finite value, or is 

0(log(F)) or is 0 (v) 

Q[m] evolves on M+, A[m] = 
R[m], same as the Berry- 
Callager tradeoff problem fT] 

P{h,s) is strictly convex in 
s,\/h 

Minimum average queue 
length is Q (previously 

known [7| but re-derived here 
using our method). 

Q[m] evolves on M+, with ad¬ 
mission control, same as the 

model in [44j 

P{h,s) is strictly convex in 

s, V/r 

Minimum average queue 
length is 0 (log (f)) 

Q[m] evolves on M+, A[m] = 
R[m], same as the Berry- 
Callager tradeoff problem [7], 
but with ergodic arrival and 

fading process 

P{h,s) is strictly convex in 

s, V/i 

Depending on the value of 

A, minimum average queue 

length either increases to only 
a finite value or is 0 (log (y)) 

Qn[m] evolves on M_|_ for ev¬ 
ery user n, no admission con¬ 
trol, same as the multiuser 

Berry-Callager tradeoff prob¬ 
lem t43j 

P{h,s) is strictly convex in 

the vector s,\/h 

Minimum average total queue 

length is individual 

average queue length is also 
0, (previously known 

[43j but re-derived here using 
our method). 

Qn[m] evolves on M_|_ for ev¬ 
ery user n, admission control 

P{h,s) is strictly convex in 
the vector s, Vfi 

Minimum average queue 
length is 0 (log (^)) 


Table 1.4: The main asymptotic results derived for discrete time queueing models with fading; unless stated 
otherwise the models assume that (v1[to]) and {H\m]) are IID 


We assume that Nc channel uses correspond to one slot. In the context of our discrete time model, 
the environment state is fixed (/iq) and c{s) = P{ho, s) is the expected number of message symbols 
which are decoded in error. The function c(s) is approximated as s times the average block error 
probability when s message symbols are transmitted using a random block code of length Nc, where 
the average block error probability is further approximated by using Gallager’s random coding upper 
bound [22l Chapter 5]. Asymptotic bounds to the optimal average delay for a given average error 
rate constraint are obtained as in Chapter 4, although in this case c(s) is a non-convex function of 
s. The asymptotic lower bounds obtained in Chapter 4 can be applied to non-convex c(s) through 
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the use of the lower convex envelope of c(s). 

In Chapter 6, we also consider a single server queueing model, where the codeword length Nc is 
a parameter for the policy, i.e., different policies can choose different codeword lengths, but every 
transmission uses codewords with the same length. For such models, it is intuitive that by using 
arbitrarily large block lengths the average message symbol error rate can be made arbitrarily close 
to zero. We show that the exponential decay rate of the average error rate with average queueing 
delay is at most where is the Gallager random coding exponent and A is the arrival 

rate of packets per slot. Furthermore for fixed length block codes, for A sufficiently close to the 
capacity of the point-to-point link, a class of fixed rate service policies is shown to achieve the 
decay rate ^Er{X). We then consider a single server queueing model where the service time can 
also be varied, to model scenarios where variable length coding is used. For variable length block 
codes which constrain the average message symbol error rate by a constant bound on the block 
error probability, the class of exhaustive service policies, which transmit all the message symbols in 
the queue at a transmission instant, is shown to achieve the decay rate ‘^Eji{X) for any A. 

We summarize the thesis in Chapter 7 and discuss some problems with scope for future work that 
are motivated by the analysis carried out in the thesis. The notation that is common to all the 
chapters in this thesis is summarized on page xii. The notation that is used in each chapter is 
summarized in each chapter. 
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CHAPTER 2 


On the tradeoff of average queue length, average service cost, and 
average utility for the state dependent M/M/1 queue: Part I 


2.1 Introduction 


We consider the tradeoff between average queue length, average service cost, and average utility for 
the continuous time single server queueing model in this chapter and the next. The mathematical 
model considered captures the problem of how a constrained/scarce resource should be dynamically 
allocated to randomly arriving demands, which may be subjected to admission control, in order 
that the system is operated optimally. Herein, this dynamic allocation problem is modelled using 
the simple state dependent M/M/1 model discussed in Chapter 1. Our primary motivation for 
modelling and studying this tradeoff problem as such, is the variety of tradeoff problems that arise 
in resource allocation problems in wireless networks. 


The state dependent M/M/1 model that we consider in this chapter is a birth death process with 
the state corresponding to the queue length, as reviewed in Chapter 1. In this chapter, we consider 
the problem of making the optimal choice of the arrival rate and the service rate at each queue 
length, such that the time average queue length is minimized subject to constraints on both the 
time average service cost and the time average utility, associated with the service of customers. We 
recall that such problems have been analysed in [70], [25], [3|, and [5j, the results of which have 
been discussed in Chapter 1. But unlike the approach in these papers, in this chapter we obtain an 
asymptotic characterization of the tradeoff in the regime 5?. The asymptotic characterization of the 
tradeoff is discussed in this chapter and the next. The notation that we use in these two chapters 


are summarized in Table 2.1 We first summarize the methodology that is used for obtaining the 
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asymptotic bounds. 


2.1.1 Methodology 

We note that if 7r(g) is the stationary distribution of the queue length for a policy 7, then 
Q(j) = EjrQ. Suppose = q} is any upper bound on 7r(g). If ^ is the largest q such that 

Pru{Q < 5 } < 5 , then > |. We obtain Pru{Q = q} for different cases to obtain q. For 
example, if c(^) is piecewise linear then it can be shown that {Q < ( 7 } = 7r(0)/3'?, where p > 1. 
We note that then g is a function of 7r(0). However, for the tradeoff problem that we consider, it can 
be shown that 7r(0) is proportional to Cc — c(A), i.e. the difference between the average service cost 
constraint and the infimum of the average service cost. Then, | > a constant x log -c(A) ) 
therefore so is ( 5 ( 7 ). This leads to the 0, ^log -c(a) )) asymptotic lower bound for a particular 
case. 

For deriving asymptotic upper bounds, we consider a sequence of admissible policies. For a par¬ 
ticular policy in the sequence, the derivation of asymptotic upper bounds on average service cost 
rate and average utility rate uses upper bounds on stationary probability distribution of the queue 
length, whereas in all except one case asymptotic upper bounds on the average queue length are 
derived using the Lyapunov comparison theorem [36] Theorem A.4.3]. For several cases, we identify 
sequences of order-optimal admissible policies 7^, for which the asymptotic growth rates of Q{^k) 
matches with the corresponding asymptotic lower bounds. Now we will discuss the model that we 
consider in this chapter. 


2.1.2 System model 


The queue evolves in continuous time, which is denoted by f G M+. The number of customers in the 
queue at time t (including the one in service, if any) is denoted by Q{t) G Z+. The state dependent 
M/M/1 model for the process Q{t) is a birth death process with birth rate = X{q), death 

rate Tq^q^i = ^{q) for q G {I,---}, and birth rate when there are zero customers in the queue, 
^ 0,1 = -^(0)- The state transition diagram of the birth-death process for a policy 7 is shown in 
Figure 2T A policy 7 is the sequence (/i(0) = 0, A(0), 7^(1), A(l) • • •) Q The set of all policies is 


denoted as T. 


We associate an utility rate function u{.) with the arrival of customers and a cost rate function 
c(.) with their service. The utility rate function models the benefit in serving customers, while the 
cost rate function models the cost incurred in serving customers. We assume that utility is accrued 


^We note that we are restricting to policies which are stationary. Such a restriction is reasonable for the class of 
tradeoff problems that we are interested in. 
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Symbol 

Description 

t 

time index 

Q{t) 

queue length at time t 

Kq) 

service rate used at queue length q 

Hq) 

arrival rate used at queue length q 

^91.92 

transition rate from state qi to q 2 for a CTMC 

u{.) 

utility rate function 

c(.) 

service cost rate function 

X, 

set of all possible service rates 

Xx 

set of all possible arrival rates 

[f a,min ; ^ a,max] 

range of values for X{q) 

[0, Tmax] 

range of values for ij,{q) 

7 

a policy 

r 

set of all policies 

Q{i) 

average queue length 

C{i) 

average service cost rate 

[7(7) 

average utility rate 

Ta 

set of admissible policies 

"TT 

stationary probability distribution for policy 7 


set of all mixtures of admissible policies 

7m 

a mixture policy 

Uc 

constraint on average utility rate 

Cc 

constraint on average service cost rate 

3? 

asymptotic regime in which Cc — )• u~^{uc) 

Q ^c) 

minimum average queue length over Ta^u under constraints Cc and Uc 

Q*(Cc,Uc) 

minimum average queue length over Fq under constraints Cc and Uc 

Pc, Pu 

non-negative Lagrange multipliers corresponding to service cost and utility constraints 

QU 

set of constraint value pairs {cc,Uc) for which admissible policies are optimal 

7*(Cc,ttc) 

optimal policy for constraints Cc and Uc 

Q*{Cc) 

minimum average queue length for the set Fq service cost constraint Cc 


stationary probability of service rate 


largest service rate < A at which the slope of c(.) changes 

l^u 

smallest service rate > A at which the slope of c(.) changes 

7?m(cc) 

minimum average queue length for the set Fq m under service cost constraint Cc 

Pi{-} 

lower bound on probability of an event 

Pn{.} 

upper bound on probability of an event 


Table 2.1: Notation used in Chapters 2 and 3. 


23 




A(0) A(l) A(2) 


Hq) ^( 3 + 1 ) 



/i(l) /i(2) /i(3) /i(g) /x(g+l) /i(g + 2) 

Figure 2.1: The birth death process under a particular policy 7 

at the rate of u{X{Q{t))) at time t and cost is incurred at the rate of c{^{Q{t))) at time t. The 

functions u{.) and c(.) are assumed to satisfy the following properties : 

U1 : The function u(A) : Xx —t M_|_ is strictly increasing and concave in A, with tt(0) = 0 and Xx 
the set of arrival rates. 

Cl : The function c{fi) : X^ —)■ M+ is strictly increasing and convex in with c(0) = 0 and 
the set of service rates. 

The set Xx is assumed to be either a) a finite set of discrete points (Aq, Ai,..., Ai^) or b) an 
interval [ra,minXa,max] of the real line. Similarly the set X^ is assumed to be either a) a finite set 
of discrete points (^uQ) • • • > /^it) oi' b) an interval rmax] of the real line. If the set Xx is a 

set of discrete points, then we extend the definition of u{X) to [ra,min = Xo,ra,max = Xk] by linear 
interpolation. A similar extension is done for the function c{fi). Note that the linear interpolation 

preserves the concavity of u(.) and the convexity of c(.). Let u~^ : M+ —> [ra,min,ra,max] and 

c~^ : M+ —)■ [rminXmax] be the inverse functions of u{.) and c(.) respectively. 

The average service cost for the policy 7 , C'( 7 ) is defined as 

^( 7 ) = limsup^E f c{fi{Q{t)))dt Q{0) = qo . (2.1) 

r^>oo -L L^o 

The average utility for the policy 7 , U{^) is defined as 

[ 7 ( 7 ) = limsup [ u{X{Q{t)))dt Q{0) = Qo . (2.2) 

T^oo L7o 

The average queue length for the policy 7 , < 5 ( 7 ) is defined as 

( 5 ( 7 ) = limsup ;^E [ Q{t)dt Q{0) = go . (2.3) 

T—>-00 ^ \_Jo J 

In this chapter, we restrict attention to policies for which the above three performance measures are 
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independent of the initial state qo, hence in the above definitions the dependence of these quantities 
on Qo is not made explicit. We note that the above definition of average utility (as in [5j) is much 
more general and encompasses scenarios where the utility of average throughput is of interest (e.g. 
as in ||44j). 

We note that the state dependent M/M/1 model can be directly applied to study resource allocation 
in modern high rate data networks. We consider such a motivational example in the next section. 
Thus, the tradeoff problem for the state dependent M/M/1 model can be studied in its own right. 
Furthermore, in Chapters 3 and 4, we shall see that the ideas developed for this simple state 
dependent M/M/1 model can be used in the study of discrete time queueing models, which in 
some cases are more representative of the resource allocation problems in wireless networks. 


2.1.3 A motivational example 

We discuss a motivating example in this section, which is based on the problem considered by Borst 
|I3j. We consider the downlink of a base station, operating in slotted time, with each slot of duration 
1.67ms. Flows, each a file of size 480776, arrive at the downlink scheduler queue for transmission 
to different users. We assume that at most one flow arrives in a slot, and the flow arrival process 
is an IID Bernoulli process. Each flow is destined to an user, which is one of two types, T1 or T2 
with uniform probability. The base station uses a round robin scheduler, which transmits bits from 
each flow, in order of their arrival instants. We assume that, if the transmitter is transmitting at 
a power level of lOlT, then the transmission rate is 50Kb/s for T1 flows, while for T2 flows, it is 
150776/s. Let ri = 50Kb/s and r 2 = 150776/s. The transmitter may dynamically vary its power to 
change a multiplier m of the transmission rate whenever a flow arrives or a flow leaves the system. 
The transmitter may choose m G A4 = {0, 0.25, 0.5, 0.75,1,1.25,1.5,1.75, 2}. The multiplier m 
is used to model the constraint that each flow may choose a rate corresponding to the choice of a 
codebook from a finite set of codebooks. Then the transmission rate is mri and mr 2 for T1 and T2 
flows. As in Borst jT^I, we assume that the transmission rate as a function of the received SNR is 
8001og^o(l + 'S'-^^)-f^V'5- We note that this formula for the transmission rate models a case where 
the fading gain is fixed or a case where transmission is done at a fixed rate only if the fading gain 
is above a certain threshold. Then the transmitter power is (m) = 64.6 ^lo( 800 ) — and 

P'^{m) = 18.5 ^10^ 800 ) — 1^ W, when transmitting to receivers with T1 and T2 flows respectively. 
We note that as a function of the number of T1 and T2 flows, and therefore the total number of 
flows in the system, the actual transmitter power used varies within a round-robin scheduling cycle. 
We are interested in dynamically controlling m as a function of the current total number of flows 
to minimize the average number of flows in the system (or for a fixed arrival rate, the average flow 
transfer latency) subject to a constraint on the average transmitter power. 

As in Borst [I3] we note that, as the minimum flow transfer time ( 300 ^/^ ) of the 
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order of seconds while the slot duration is of the order of milliseconds, we can model the system 
as a M/G/l-PS (processor sharing) queue, but with control on the total rate of service, through 
the choice of m as a function m{q) of the current number of flows q in the system. The Bernoulli 
arrival process is approximated as a Poisson process of rate A. Let us consider the normalized 
service requirement for T1 and T2 flows. If a Tl-flow is the only flow present in the system, then 


it requires a time of if m = 1. Similarly a T2-flow requires a time of jDjs if m = 1. So the 


normalized service requirement of the flows arriving are distributed as ^ with probability 0.5 and 
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^ with probability 0.5. The time sharing amongst users manifests itself as processor sharing in 
the continuous time model. At a time t, if there are q flows in the M/G/l-PS model, the remaining 
service requirement of each flow is reduced at the rate From Bonald jT!] Theorem 2], we 

note that for any policy 7, the stationary probability of the M/G/l-PS queue with control on the 
service rate, is independent of the service requirement distribution. Therefore, following Borst |T3j, 
we consider a M/M/l-PS queue with control on the service rate, where the service requirement 
distribution is exponential, but with the same mean as the service requirement distribution in the 
M/G/l-PS queue, i.e, a mean service requirement of ^ = 6.4s. At a time t, if there 

are q flows in this model, the remaining service requirement of each flow is reduced at the rate 
^^1^. Therefore the rate at which a flow leaves the system is We then note that the 

M/M/l-PS queue with state dependent total service rate, determined by the policy 7, is the same 


as the model considered in Section 2.1.2 The number of flows in the system is modelled by the 
state of the birth-death process. The birth rate is A, while the death rate which is dependent on q 
can take values in 5 = x {0,0.25,0.50,0.75,1,1.25,1.50,1.75,2}. 


We note that the average transmitter power used when there are q flows in the system depends on 
the proportion of T1 flows and T2 flows. If there are qi T1 flows when there are q flows in the 
system, then the average transmitter power is ^ Jq obtain good policies which 

tradeoff average power with average number of flows for the discrete time system, we analyse the 
M/M/l-PS model, under the assumption that the transmitter power is P(m) = P^{m), irrespective 
of the flow which is being transmitted. The model obtained under this assumption is denoted as 
M/M/l-PS (I). We also repeat the analysis under the assumption that P{m) = P‘^{m), in which 
case the model is denoted as M/M/l-PS (II). The results obtained from the analysis of both 
M/M/l-PS (I) and M/M/l-PS (II) models are found to be useful in designing policies for the 
discrete time system. 


2.1.4 Overview 

We formulate the tradeoff problem in Section [2^ for a restricted class of admissible policies. A 
non-idling property of any optimal admissible policy is also shown in the same section. We consider 


26 








three cases of the tradeoff problem: FINITE-^uCHOICE, INTERVAL-/iCHOICE, and INTERVAL- 
A/xCHOICE, which correspond to different choices of the sets and The analysis of FINITE- 
/iCHOICE in the asymptotic regime 3? is carried out in Section while INTERVAL-/iCHOICE, 
and INTERVAL-A/xCHOICE are analysed in Chapter 3. For FINITE-//CHOICE, where TA = {A}, 
we identify three cases based on the value of A and the set of available service rates {/uq, ■ ■ ■ ,/^k}, 
for which the asymptotic behaviour of the tradeoff curve in the asymptotic regime 3? is different. 
Asymptotic lower bounds and upper bounds to the tradeoff problem for these three cases are then 
obtained in Section [2.3.1 and Section 2.3.2 respectively. An asymptotic characterization of optimal 
policies is presented in Section [2.3.3 We then numerically illustrate the asymptotic behaviour of 
the solution to FINITE-/iCFIOICE for several examples in Section 2.3.4 Asymptotic bounds to the 
tradeoff curve for the example in Section 2.1.3 are then presented in Section [2.3.5| using the results 
derived in this chapter. 


2.2 Problem formulation 


In this chapter and the next, we consider the tradeoff problems ( |l.ip and (1.3) for a restricted class 
of admissible policies T^. The set of admissible policies is defined as follows. 


Stability : A policy 7 is defined to be stable if the birth death process Q(t) under policy 7 is 
irreducible and positive recurrent with stationary distribution 

Admissibility : A policy 7 is admissible, if 


G1 : it is stable. 


G2 : the sequence (fi(0), /u(l), /j.(2), ■ ■ ■) is non-decreasing, and. 


G3 : the sequence (A(0), A(l), A(2), • • •) is non-increasing. 


Then we define the set of admissible policies as 

Ta = {7 : 7 G r, 7 is admissible}. 

Remark 2.2.1. We note that restricting attention to T^ is reasonable, as the optimal policy which 
minimizes the average queue length subject to constraints on the average service cost and average 
utility possesses the properties Gl, G2, and G3 in many cases (see Chapter 1)0 


^We note that there exists an admissible policy which achieves the minimum for the constrained optimization 
problems or ( |1.3[ ) for certain values of the constraints Cc and Uc. For example, for | |1.1[ |, these values of Cc and 
Uc are such that there exists Lagrange multipliers fSi and P 2 for which average service cost and average utility of any 
admissible optimal policy for the dual problem (|1.2[) are equal to Cc and Uc. 
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We note that for any admissible policy, we have that 


C{l) = 

C/( 7 ) = E^^«(A((5)), and 

Q{i) = 

where Q tTj and the performance measures are independent of the initial state go- We note that 
for a policy 7 if G2 and G3 holds and if at any finite q', ii{q') — A(g') = e > 0, then 7 is stable and 
therefore admissible. 

We note that a larger set of policies can be obtained by mixing the pure policies in Fa. We note 
that a mixture policy corresponds to time sharing of pure policies, with the time period, in which a 
particular pure policy is used, tending to infinity. The set of policies which are obtained by a finite 
mixture of the policies in Fa is denoted as F^ We note that associated with a 7 m £ F^ m we 
have a set F(7 m) T Fq, which is the set of policies which are mixed according to a probability mass 
function for 7 G F(7m)- For a 7m G Fa,M, Qilu) = The average service cost 

rate and average utility rate are defined similarly for 7 G F^ m- 

2.2.1 Problem 

Our objective is to solve the following optimization problem, TRADEOFF-M: 

minimize Qil) 

such that C'( 7 ) < Cc, 

and U{-i)>Uc, (2.4) 

where Cc and Uc are constraints on the average service cost and average utility respectively. The 
optimal value of the above problem is denoted by Q*j^{cc,Uc)- We note that above constrained 
minimization of the average queue length corresponds to the constrained minimization of average 
delay if the average arrival rate is fixed. 

In the following lemma, we show that TRADEOFF-M can be solved, only for certain values of Cc 
and Uc- 

Lemma 2.2.2. If TRADEOFF-M has any feasible solutions, then u~^{uc) < c“^(cc). 

Proof. Assume that there is an policy 7 m £ F^m which is feasible for TRADEOFF. Then from 
Jensen’s inequality we have that c(Ep.^ [a(Q)]]) < [c{p{Q))]] andEp.^ [E^.^ [^^(^(Q))]] < 

u{Kp^ [E^.^ [-^(Q))]]- For brevity let us denote Ep..^ [.]] by Just E [.] in this proof. There¬ 

fore E/i((5) < c~^(Ec{p{Q))) and «“^(Eii(A(Q))) < EA((5). As E^.^Q < 00 , E^.^/i((5) = 
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^iTga^m.aKQ)’ ^7 G T (7M ) ■ Therefore for 7, u ^(Ett(A((5))) < c ^(Ec(;u(Q))). From the non¬ 
decreasing properties of c(.) and u{.) we have that c“^(.) and u~^{.) are also non-decreasing. Hence 
if there is any one feasible policy 7, u~^{uc) < c“^(cc). □ 

If c“^(cc) > u~^{uc), then we show that there exists a feasible policy 7 G on a case by case basis 
in the following discussion. We note that if c“^(cc) > u~^{uc), then it is not guaranteed that an 
optimal policy J*iCc,Uc) eVa, M exists for the above problem. However, in the following discussion 
we identify a set of {cc,Uc) for which the existence of an optimal policy in Tq m is guaranteed. 

Remark 2.2.3. Let ( 3 c and ( 3 u G M+. Consider an unconstrained MDP denoted as MDP{( 3 c, I 3 u) 
as in |5i| which is obtained by uniformization at rate with single stage cost ^ . Then 

from |[5j we know that an optimal policy YiPc f^u) G T^ exists for MDP{( 3 c, ( 3 u)- Let T*{f 3 c, ( 3 u) be 
the set of all optimal Ta policies for MDP{( 3 c, ( 3 u)- Also let r^(/3c, /3u) be the set of all mixed poli¬ 
cies obtained by a finite mixture of 7 G r*(/5c,/3n). Let 0“ = {(C'(7), t/(7)),7 G r^^(/3c,/3«), V/3c, > O}. 

Then from [^|, if {cc,Uc) G C>“, then there exists an optimal policy in Ta,M for TRADEOFF-M. 

Remark 2.2.4. Suppose (cc, ttc) 0 but c“^(cc) > u~^{uc)- We note that then for all 7 m which 
are feasible for TRADEOFF-M, Qi'-fu) > 0. Hence, for every e > 0, there exists some feasible 7 m 
such that QijM) < Q*m{cc,Uc) + e. We call such policies e-optimal for Cc- 

We now show that any optimal policy for TRADEOFF-M is non-idling, if it exists. 

Lemma 2.2.5. Any optimal mixed policy 7*(cc,ttc) for TRADEOFF-M, has g,{q) > 0 for every 
q>l, for every 7 G r(7*(cc, tic))- 

Proof. Let 7 G r(7*(cc,itc)) be an admissible policy with service rate and arrival rate given by 
/j{q) and X{q) respectively for q > 0. Let qo = max{g : fi{q) = 0}. Assume that for 7, qo > 0. 

As 7 is admissible, ^{q) = 0, for all q < q^. Then the states {0 ,..., go ~ 1} 3re transient under 

policy 7. Let 7' be another policy such that at a queue length q the service rate and arrival rate 
are /u'(g) and X'{q) respectively. For Vg, let /u'(g) = ^r{q + qo) and A'(g) = A(g-Fgo)- We note 
that the birth-death process under 7' is obtained by a relabelling of the states under the policy 
7. And 7' is admissible as 7 is admissible. It is clear that U{'y') = U{'y) and C{'y') = C'(7), but 
Qil') = Q{l) — Qo- Thus any 7 such that go > 0 cannot be an element of r(7*(cc, Uc))- □ 

Thus in the following we need only consider non-idling admissible policies. 

In the following discussion we consider the problem TRADEOFF, 

minimize Q{l) 

such that C'(7) < Cc, 

and tf(7) > Uc, (2-5) 
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where we minimize over the set Fq only. The asymptotic bounds on the optimal value for TRADEOFF- 
M can be obtained easily from the analysis of TRADEOFF. We now consider three special cases of 
TRADEOFF, which are either representative of the problems that arise in the context of communi¬ 
cation networks or are useful in understanding the tradeoff for discrete time queues. 


FINITE-^uCHOICE: 

1. We restrict to policies 7 such that X{q) = X,\/q £ Z_|_. 

2. For any such policy 7 , U{'y) = u{X). We choose A such that u{X) > Uc- 

3. We also restrict to policies 7 such that //(g) £ = {//q = 0,//i,/i 2 , • • • ,//k}, where 

/ti < /Uj+i, /i/c = Traax < oo. Thus the available service rates take values from a finite 
discrete set. We assume that A < Vmax- 

4. The optimal value of the tradeoff problem is denoted by Q*{cc)- 

INTERVAL/iCHOICE: 

1. We restrict to policies 7 such that X{q) = X,\/q £ Z+. 

2. For any such policy 7 , U{j) = u{X). We choose A such that u(A) > Uc- 

3. We restrict to policies 7 such that /i(g) G [0,rmax]- Thus the available service rates take 
values in a finite interval. We assume that A < rmax- 

4. The optimal value of the tradeoff problem is denoted by Q*{cc)- 


INTERVAL-A/iCHOICE: 


1. We restrict to policies 7 such that X{q) £ [ra,min, ra,max], where ra,min > 0,ra,max < 00 . 

2. We restrict to policies 7 such that //(g) G [0,rmax]- We assume that Va^min < ’<'max- 


We note that for FINITE-//CHOICE and INTERVAL-/iCHOICE, the constraint on the average 
utility in TRADEOFF is satisfied by the choice of A, and therefore this constraint is not explicitly 


mentioned (as in (1.3)). In the following, we obtain an asymptotic characterization of Q*{cc) 
for FINITE-//CHOICE, while INTERVAL-//CHOICE and INTERVAL-A/rCHOICE are analysed in 
Chapter 3. 
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2.3 Analysis of FINITE /iCHOICE 


In the following, we state the motivation for considering FINITE-//CHOICE. 


Remark 2.3.1. FINITE-/iCHOICE is motivated by the tradeoff problem ( |1.7[ ) for the following 
discrete time queueing model. Customers arrive in a batch of random size, in every slot, into 
an infinite length queue. All the customers which arrive in a slot are admitted into the queue. 
The number of customers, which are served in each slot, or the service batch size, is chosen as 
a deterministic function, of the current queue length. This feature of the discrete time queue is 
modelled by the choice of the service rate, /^(g), as a function of q in FINITE-^uCHOICE. We assume 
that for the discrete time model, the queue evolves on the set of non-negative integers. Hence, 
the service batch size also takes values in the set of non-negative integers. The essential feature 
here is that the set of batch sizes is discrete and therefore we assume that ^{q) takes values in 
a finite discrete set. As there is no admission control in the discrete time model, we assume that 
the arrival rate is a fixed A for every q for FINITE-^uCHOICE. In each slot, assume that there is a 
service cost incurred in serving the customers. This is modelled by the service cost rate function 
c(.) in FINITE-/iCHOICE. By analysing FINITE-^CHOICE we illustrate the basic techniques which 
are used in the asymptotic analysis of TRADEOFF, which turn out to be useful in the analysis of 


problem (1.7). 


We now present an asymptotic analysis of FINITE-/iCHOICE in the regime 3?. We note that 
part of this analysis was presented in |62]. For brevity, we use tt rather than tt-j to denote the 
stationary distribution corresponding to an admissible policy 7 in places where there is no source for 
confusion. The stationary probability of queue length being q is denoted by 'n'{q). The stationary 
probability of using a rate //fc is denoted by 7r^(A:). We note that vr^(/c) = 

^*( 7 ) = Since c(/x) is convex in jj., by Jensen's inequality, we have that the 

average service cost C{'y) > c(A), for any admissible policy 7. We note that c(A) is the minimum 
average service cost which has to be expended for the average service rate to be equal to the average 
arrival rate, as noted in Chapter 1. 

We first obtain an asymptotic lower bound to Q*{cc) in the regime as Cc i c(A), by finding a lower 
bound on Qi'jk), as a function of C{'yk) — c(A), for any sequence of feasible admissible policies 7 ^ 
with C{'yk) i c(A). Subsequently, we show that there exists a sequence of admissible policies 7 ^ for 
which C{'yk) approaches c(A) arbitrarily closely, so that c(A) = inf.ygra 

The asymptotic behaviour of Q*{cc) for FINITE-^CHOICE depends on the behaviour of c(/i) in 
the neighbourhood of /i = A. We now define quantities ku, Hu and ki, which are related to this 
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behaviour. 


— 




mm 


: k < K - X, 


c(pk+l)-c{f^k) y c(/Jfc)-c(^fc- 


IJ-k + l—lJ-k 


f^k f-^k 






ma*{„ : t > 1,M. < A, 


if this set is non-empty, 
otherwise. 

if this set is non-empty, 
otherwise. 


Let fiku = l^-u and fiki = W- In words, fiu is the service rate /xa:„ greater than or equal to A at which 
the slope of c(.), considered for service rates in X^, changes. A similar interpretation can be given 
for Hi. Depending upon the value of hi, I^u, and their relationship with A, there are three different 


cases that need to be considered (also see Figure 2.2): 


FINITE-;uCHOICE-l : Hl = ^A<^^u, 
FINITE-;uCHOICE-2 : w > < A < Hu, and 

FINITE-/iCHOICE-3 : hi ^ = A = Hu- 



Figure 2.2: Illustration of the relationship between A, hi, and Hu along with the minimum average service 
cost c(A) and the line 1{h) for the three cases that arises for FINITE-/iCHOICE problem 


We now state the motivation for this classification. We note that as the constraint Cc approaches 
c(A), if there exists a feasible policy 7 with C{'y) < Cc, then for that policy the stationary probability 
of certain service rates should go to zero. The classification is based on the set of service rates, 
whose stationary probability goes to zero. For example, for FINITE-/iCHOICE-3, as (A,c(A)) is 
a corner point, as Cc i c(A), the stationary probability that any service rate other than A is used 
approaches zero. For FINITE-/iCHOICE-l and FINITE-/iCHOICE-2, as Cc i c(A), the stationary 
probability that any service rate which is less than hi greater than Hu is used, approaches zero. 
We note that > A in the case of FINITE-//CHOICE-1 and FINITE-^CHOICE-2, and as Cc i c(A) 
the service rate Hu > ^ could be used, unlike in the case of FINITE-^CHOICE-3 where only // = A 
can be used. Furthermore we note that for FINITE-^CHOICE-1, = 0 and for both FINITE- 
;uCH0ICE-2 and FINITE-/iCHOICE-3, hi > 0. Then as Cc i c(A), for FINITE-/iCHOICE-l, for a 
non-idling 7 G Tq, the queue becomes empty infinitely often, while this cannot happen for both 
FINITE-/iCHOICE-2 and FINITE-//CHOICE-3. 
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2.3.1 Asymptotic lower bounds 


For an admissible policy 7 , for obtaining an asymptotic lower bound on Qij) we: a) obtain an upper 
bound on Tr^{k) for certain values of k in terms of ( 7 ( 7 ) and c(A), b) relate the stationary probability 
7r^(A:) to the stationary probability of the queue vr(g), and c) obtain a lower bound on < 3 ( 7 ) in terms 
of iT{q). For the cases FINITE-/iCHOICE-l and FINITE-//CH0ICE-2, define the line /(^) as the 
line through the points and {ij,u,c{^u))- For the case FINITE-/iCHOICE-3, let l{fi) be 

any line through (A,c(A)) with slope greater than - and less than Jhe 




Mfe;+1 


-A 


line l{^) is illustrated for the three cases in Figure 2.2 We note that c(A) = l{\) by construction. 
Furthermore 1{^{Q)) = 1{X) as I is linear. 


We now present an upper bound on the stationary probability of certain service rates as the average 
service cost approaches c(A). 


Lemma 2.3.2. Let TZk = {k \ Hk < W or k‘k > k^u}- For an admissible policy 7 , for all k G TZk, 




Cil)-ciX) _ 

c{kk) - Kkk)' 


Proof. We have that (7(7) - c(A) = Ef=o“ Kkk)]- For k G c{kk) - Kkk) = 0. 
Since c{kk) > l{kk), for all k G TZk, we have that vr^(A:) < ^ 

A non-idling admissible policy 7 is specified by the sequence (go = 0,gi,g2,-'‘ ,<1k-i,Qk = 
00 ), Qk < Qk+i, which is such that 

kiQo) = a ( 0 ) = 0 

k{q) = /Ufc, if g G {gfc_i-F 1, • • • Wfc}, for A: G {1, • • • ,i7}. 

For any k, if qk-i = qk, then the rate kk is not used by the policy 7 . From the definition of vr^(A:) 
we have that 


7rM(0) = 7r(0), 

Qk 

= £ 7r(g),Vfc>l. (2.6) 

9=<?fc-l+l 


In the following we obtain lower bounds on the average queue length < 5 ( 7 ) as a function of the 


upper bounds in Lemma 2.3.2 on stationary probabilities of service rates, when Cc — c(A) = fF | 0 for 
the three cases FINITE-^CHOICE-1, FINITE-/xCHOICE-2, and FINITE-^CHOICE-3 respectively. 


We first consider the case FINITE-/iCHOICE-l, where ki = 0 and ki = 0- 
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Remark 2.3.3. If fXu = IJ-K, then we note that c(/u) is a linear function of Then the admissible 
policy 7 , with //(O) = 0, and ^{q) = hk for all q > 1, has C{'j) = Ec(//((5)) = c{Efi{Q)) = c(A). 
Furthermore, 7 has the minimum average queue length ■ Hence in this case there is no tradeoff 
between the average queue length and average service cost. So in the following, we assume c{^) is 
such that fiu < fJ'K- 

We note that an admissible policy 7 , which uses only the service rates G {0, • • • ,ku}} has 

Cil) = Efc=o^/^(^)c(/^fc) = Y!k=o^Pi{k)Kk'k) = c(A). Furthermore, ^(7) = c(A) is achieved only 
by admissible policies which uses only the service rates {/x^, /c G {0, • • • , /?«}}■ Hence, as A < Hu, 
the policy 'ju, that uses ;u(0) = 0 and ^{q) = fiu for q > 1 has a service cost of c(A). We note 
that 7 n has the minimum average queue length ^ , among all policies 7 for which C{j) = c(A). 

Thus the minimum average queue length, among policies 7 for which C{'y) < Cc, where Cc > c(A) 
is at most ^ We note that if C{j) > c(A), then service rates /x^ with k > ku could be used, 
which could yield an average queue length less than ^ . In the following lower bound on Q{^), 

we observe that if Cc > c(A), then the average queue length can be less than ^ , but has the 

limit as C( 7 ) ^ c(A). 

Lemma 2.3.4. For any sequence of non-idling admissible policies 7 fc with C( 7 fc) - c(A) = 14 i 0, 
we have that 


In the proof, for any sequence of non-idling admissible policies 7 ^, we first show that as Vk = 
C{lk) “c(A) 4 , 0 , the largest queue length qj.^ at which service rate is used, increases to infinity 
as log This asymptotic lower bound is obtained by showing that a lower bound qk^,i to qk^ 

increases as log 3 T^, the lower bound qk^^i is used to define the policy 74., which has 

fi{0) = 0,iJ,{q) = Hu for 1 < q < qk^,i, and = hk for q > qk^^i- We note that for any policy 
7fc with C'(7) — c(A) = I4, Qi'jk) > Qil'k)- The sequence Qi^'k): obtained as I4 i 0 , is shown to 
have the asymptotic behaviour in the above lemma. 


Proof. Let us consider a particular policy 7 in the sequence 7 ^ with C'( 7 ) — c(A) = V. As Q{t) is 
a birth-death process we have that 

7r{q- 1) < 7r(g)y,Vg G {I,-" ,qkj- 

By induction, we obtain that 

- m) < 7r(gfcJ (y) < 7r(gfc„-F 1)^ (y) , Vm G {0, • • • , 
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Now we note that 




q=0 

Qku 

but '^Tr{q) 


K 


I^U 

'^TT^{k) = 1- TTf,{k) 

’ " k=ku+l 


k=0 

gk 


q=0 


Qku Qku 

Y - m) < TT{qku + 1)^ X] 

m=0 m=0 


A 


K K 

and from Lemma 12.3.21 1 — >1-5: 

k=ku-\-^ k=ku-\-^ 


V 


c(/Uk) - l{fQk) ' 


Hence 


or 


or 1 - 

k=kii-\-l 

Hu A 1 
HK T^{qku + 1 ) 


J+1 - l{Hk) “ A ( A ) 

K y 

Y ~( —^— u —f - 

^yk'k) -i{Hk) Hk 

y -A/7/-r± 


- 1 


K 


V 


k^+1 


Hku A 

+ i< 


Therefore 


Qku > 


log(^) 


Hu A 1 
HK T^{qku + 1 ) 


K 

1- E 


V 


fcJC+i ~ 


- 1. 


But we note that Tr{qk^ + 1) < Ef=fc„+i < Ef= 


^ = —. So that 


Qku > log(i^) 


Hu A / ^ ^ 


ku+l c{tik)-l{tkk) Cl 

- 1. 


HK 

Therefore, for any non-idling admissible policy 7 we have that q^u > Qku,i' where 


Qku,I = 


log(-i 


Hu A Cl 
M-ft: 


i-L 

Cl 


+ 1 


- 1 


(2.7) 


Now we note that for the policy 7 under consideration, p,{q) < for q G {0, • • • ,qku}< 
h{q) < HK for q G {qku + 1, •'' }■ Let policy 7' be defined as follows : 


n'iq) = ^ 


0 if g = 0, 

/ill if g G {I,-- - ,qku,i}, 

HK otherwise. 


We note that Q{'^') < <3(7)- Let the stationary distribution of CTMC under 7' be denoted as 
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7r'(g). Let a = — and b = —. Then we have that 

Mu Mif 


7r'(g) = 


7r'(0)a'? if g G {0, • • • 

otherwise. 


We note that 7 ' is also admissible. As 


vr'(O) 

vr'(O) 


Qku,l 


g=l g=gj.^;+l 

1 H- - — (1 — ^— 

1 — a 1 — 0 


= 1 , 


= 1 . 


We note that vr'(O) can be obtained from (2.8) in terms of We have 


< 3(7 ) = 

g=0 




9=1 9 =<?)c„,! + l 


Simplification leads to 


( 2 . 8 ) 


Q(y) = 7r'(0) 


((1 - a)(l - + 1)0'?'““'') + a(l - a‘"=“'')) + qk^^i- -r + 


(TiV + 1)0'^''“'') + a(l - a«''-')) + a'?''-' + (j^) 


l+T^(l-a«'=u.0 + a«'=-'T^ 


(2.9) 


where 7 r'( 0 ) was obtained from (2.8). At this point, we note that as F —)• 0, —)> 00 and 

this lower bound < 5 ( 7 ') to Q(j) approaches which is what we expect. However, in order to 
ascertain the behaviour of average queue length as V approaches zero, we need to lower bound the 


right-hand side of (2.9) for 17 > 0. 


The denominator of (2.9) can be bounded above as follows : 


1 + 


“ « ou , b n, , 

- — - QHkii,l _j_ - (2"ku,l 

1 — a 1 — a 1 — 6 


a 


= In-h 

1 — a 


1 


1 


i-1 i-1 


As 6 < a, < T^. so that the denominator of (2.9) < After substituting this upper 
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bound for the denominator in (2.9), we have 


< 3 ( 7 ') > (1 - a) 


(1-a)^ 


{1 - (1 - a){qk^^i + 1 ) 


_ n _L 


— a.a 


} + a«'=“'' - - + 


a 


> 


1 — a 
a 


+ (l-a) <qk^^i 


1 — 6 (1 — 6)2 j (1 — a) 


1-6 ( 1 - 6)2 


{(1 - a){qk^^i + 1 ) 0 '^''“’' + 


1 — a 1 — a 


[1 + (1 _ a)(7/c„,;]. 


( 2 . 10 ) 


From (2.7), with a < 1, we have that 




loga 
a L 


QF A f 




loga(l/“) 


^ (ci - y) + F' 






From the definition of i we have that 


qku,i < ^°S(^) 


^iu A ^ ^ ^ 


flK 


Therefore, + (1 — a)qku,i) ^ 




MiC 


1 + (1 — Cl) 


l^u / Cl 






Substituting this in (2.10) we have that — Q{'y') < 


a y 


1 — a 1 a Atii-A , 


c, (l-S)+V 


1 + (1 — g) 


Mit \ 

A ^ 


f-^u / Cl 


L \v 




( 2 . 11 ) 


Thus, for a sequence 7 ^ such that I 4 i 0, we have that — Qi'jk) = O iVk log ( ^ 


□ 


Corollary 2.3.5. For any sequence of non-idling admissible policies 7 m, fc £ ^a,M with C{'jM,k) 
c(A) = 14 1 0, we have that 


^-e(7«,*) = o(v.i„g(i 



Proof. For a k, if 7M,fc is such that C{'^M,k)— c{\) = 14, then we have that for every 7^ G r(7M,fc), 
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C{ 7 k)-ciX)<^ = U^,. Then 


P-yk 


^ -Q{^,) = o(u^,log(j^ 


IJ‘U X 


a 


Ik 


We note that Q{^k,M) ~ ^p-y^Qilk)- Then applying to LHS and RHS of the above equation 
we obtain that 


A 


X 


Qilk,M) = '^Plk 


o I u,, log I tA 


a 


Ik 


We note that by definition, there exists some constant c > 0 such that the RHS is 

1 


< 




U^k log 




7fc / J J 


Since the function ?xlog (:^) is concave, we have that 


Y.P'Tk 




< 


cY^Plk [1^7 J log ( 


xYlPik [^7fe] 


Since [U^J = Vk we have that 


^-Q(7M,.) = o(v.log(l 


□ 


We now present asymptotic lower bounds for FINITE-^CHOICE-2 and FINITE-/iCHOICE-3. For 
an admissible policy 7 , to relate the stationary probability distribution 7r{q) to the average queue 


length ( 5 ( 7 ), as noted in Section 2.1.1 we make use of the fact that if q is such that Ylq=o — h 


then < 3 ( 7 ) > |. The choice of ^ here is arbitrary. The best lower bound on ( 5 ( 7 ) is given by the 
largest q such that Ylq=o'^(^) — 5- 


Lemma 2.3.6. For any sequence of non-idling admissible policies 7 ^ with C{'yk) — c{X) = 14 J, 0, 
we have that 

n (^log for FINITE-/XCHOICE-2, 

for FINITE-/iCHOICE-3. 


Qilk) = < 


( 2 . 12 ) 


Proof. We note that ki > 1 for the cases FINITE-/xCHOICE-2 and FINITE-//CHOICE-3. Con¬ 
sider a particular policy 7 in the given sequence 7 ^, with C{'y) — c(X) = V. Then we note 

.0 = 7 fram the upper bound in Lemma 


tlta* < Et 


h-l 


fore, Also, for every q e {0, • • • vr(g) < Now 


^h-i 


2.3.2 


There- 


we intend to find the largest q such that Y11=o'^{q) — 5 - ^^it as Yfq=o^ ^ 


nd 
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7r{qkt-i + 1 ) < 7 r(gfc,_i)^ < the largest such q satisfies 


g=<?fe;-l + l 


V 

Cl 


In the following, we use an upper bound on 7r(q) which leads to a lower bound q^ on q. Since 
7r(q)A = 7r(q + l)/u(q), and for q> qui, ^ (from the admissibility of 7 ), we have that 


q Q 

7r(g) < 7r(^fc,-i) Y 
q=qkj-i + l m=l 

If is the largest integer such that 




(2.13) 


then < 5 and g > Qi. 


In the case FINITE-^CHOICE-2, A > m, so that summing the geometric series in (2.13), we have 
that 


x-fii (i y' 

- 1 < ' , \ -^ I or 

Wy A7r(gfc,_i) V2 q. 


qi < qki-i + 


log/^ 

\h 


1 + 


(I V 


and, in fact. 


Qi = 


qki-i+ log 


Mi 


1 + 


A7r(qki-i) \2 c'j 


\-m /1 V 


XTr{qki-i) \2 c'l 


Since qk,-i > 0, we have that 
Qi > log 




\ A7r(gfci-i) V2 4 

Now since TT{qk,-i) < 7^^,ikl) < have that 

(A-w)(c(Aifc,-i)-/(^fci-i)) A V 


qi > logr^'j ( 1 + 


AV 


2 c[ 


TT - ^ - 1 


Then we have that 




(2.14) 


(2.15) 
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Thus for FINITE-/iCH0ICE-2, we have that Qi'jk) = ^log ■ 

For FINITE-/iCFI0ICE-3, A = ///, so that from (2.13), instead of (2.14) we have that 


qi - qki-i < 


1 V 


7r(qki-i) \2 c\j ' 


and, in fact. 


qi = 


qki-i + 


1 /I V 


TT{qki-i) \2 d^) 


Proceeding as for FINITE-//CFI0ICE-2, we have that 

c(/xfc,_i) - l{fiki-i) A V 


qi > 


V 


2 d 


Therefore we have that 


Q(7)>l>f>-, 


^ ^ c(^ki -l) - f 1 V 


V 


2 c 


(2.16) 


Flence for FINITE-//CFI0ICE-3, we conclude that Q{^k) = Ai 


□ 


Corollary 2.3.7. For any sequence of non-idling admissible policies ^k,M £ ^a,M with C{'jk) — 
c(A) = Vfc I 0, we have that 


Q{lk,M) 


/ 

n (log for FINITE-/iCHOICE-2, 

Q for FINITE-/iCHOICE-3. 


(2.17) 


The proof of this corollary is very similar 
property of the functions log (^) and are 


to that of Corollary 2.3.5| except that the convexity 
used instead of the concavity of the function txlog (^). 


2.3.2 Asymptotic characterization of Qlf(cc) 

In this section we obtain asymptotic upper bounds for TRADEOFF-M. The sequence of policies 
that is constructed for FINITE-/iCHOICE-l is motivated by the policy 7 ' that was used in the proof 
of the lower bound. We shall see that a sequence of policies with qk^ scaling as log (A gives the 
correct asymptotic upper bound. 

Lemma 2.3.8. For FINITE-//CHOICE-1, there exists a sequence of non-idling admissible policies 
7 fc with a sequence 14 I 0 such that “ Qidk) = © (l41og (\^)) and Ci'jk) - c(A) = 14- 
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Proof. We first consider a policy 7 in the sequence of policies 7 ^. The policy 7 is defined as follows: 

p{0) = 0 , 

p{q) = Pu, for qe {I,...,qkj, 
p{q) = PK, for q £ {qk^ + 1,... 


The sequence of policies 7 fc,A: > 1 is obtained by choosing q^^ = k. Consider the policy 7 . As 
= 1 we have that 


7r(0) 


Pu-^\ \PuJ ) \Pu) PK - 


= 1 


Therefore, we obtain that 


7r(0) = 


1 + 77 ^+ 


/ \ 'Jfeu 

A 

X 

VMu/ 

JlK-^ 

t-f’U 


Y, and 


vr, 


iiK) = 7r{q) 

'J=<?feu+i 


A \ A 


Mu 7 Mk-A 


/ A'Jfeu 

A 

X 

Vmu/ 




(2.18) 


From the definition of C{'y) we have that 


Cil) 


C( 7 ) - c(A) 


T^lJi{ku)c{pu) + TTij.{K)c{flK), 

(c(A) + mi{fiu - A)) + 7r^(Ar) (c(A) + m 2 {pK - A)), 
c(A)(l - 7r(0)) + miinu - A)7r^(A:«) + m 2 {pK - \)Tr^,{K), 
c(A) - 7r(0)c(A) + - A)(l - 7r^(iC) - 7r(0)) + m 2 {pK - A)7r^(iC). 

miipu - A) + 7r^(iC) [m 2 {pK - A) - - A)] - 7r(0) [c(A) + - A)], 

(2.19) 


where mi = 
c(A) + mi(/i„ 
written as 


c(uu)—c(X) 1 c(ux)—c(A) 

A ’ and m 2 = A ' 

fj-u — X ^ MX —A 

- A) = c{fiu) and m 2 {pK - A) - 


We denote (^( 7 ) — c(A) by V. We note that 


7«i(/7m — A) = c{^k) — c{fiu)- Then (2.19) can be 


V 

V - - A) 


- A) + 7 r^(iC) (c(//ic) - c{nu)) - vr(0)c(//„), 
- c(/7u)) - C{flu) 


/ A 

A 

A 

Vmu/ 

Mx-A 

1 

1 
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Simplifying, we obtain, with c(A) = miA, 


[17 - miifiu - A)] 

1 + ^ A 

+ cinu) 

icinx) c{nu)) 

A A 

/-I'K — ^ t^u — ^ 

-A Hu-A 

(17 - miipu - A)) 

) -h miifiu - A)) 


A(m 2 - mi) - ( ^J (C( 7 ) - c(A)) 


A V'‘“ 

Hu J 

A ^ 

I^U y 


SO that, 


Qku — 


A(m2 - mi) - ^ 


A 


Hx —A Hu—A 








We note that the average queue length for the policy 7 is of the form given in (2.9), wherein a = 


and b = Simplifying this expression, we obtain that 


Q{i) = 


(T^ + 


T^-T^ +a9.u 


( 1 - 6 )^ (l-a )2 


1-^^—h 

1—a ' 


b _ Q 

1—6 1—a 


( 2 . 20 ) 


We are interested in only order approximations of < 3 ( 7 ); so we proceed by considering large k, so 
that y I 0 and J, 0. We obtain that (only the dominant terms as ^ J, 0) 


Q(7) 


1 — a 


+ 9fc„a'^''“(l - a) 


b a 

1—6 1—a 


+ (1 — a) 

X ( 1 + 0 '^'=“ (1 — a) 


6 a 

(1 — 6)2 (1 — a)2 

a 6 


1—a 1—6 


Expanding, we obtain 


1—a 1—a 


1—a 1—6 


+ 9fcua^''“(l - a) 
+a'^'““ (1 — a) 


-9fcua^^'‘“(l - o)^ 


a 6 

1—a 1—6 


1 2 


+ (1 _ a )2 


1—6 1—a 

6 a 

(1 — 6)2 (1 — a)2' 

6 a 

1 — a 1 — 6 (1 — 6)2 


( 2 . 21 ) 

( 2 . 22 ) 

(2.23) 


We note that the second term in (2.21) is positive and 0(17), while the third term is negative and 


0(l71og(y)). The term in (2.22) is negative and 0(17). The first term in (2.23) is negative 
and 0 (172 log , while the second term is negative and 0(172). i-|.^g dominating term 

in — < 5 ( 7 ) is positive and 0(yiog(y)) as 17 | 0. Hence we have that — Q{'y) = 

e{v'iog(f)). 


We note that every 7 ^ constructed, by choosing = k, is non-idling and admissible. So, there 
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exists a sequence of non-idling admissible policies 7 ^ such that ^ — Qi'jk) 

and 0 ( 7 ) - c(A) = Vfc. 


e(F.iog(^)) 

□ 


Using the asymptotic lower bound on Qi'jk) from Corollary 2.3.5 and the above asymptotic upper 
bound, we have the following result. 


Proposition 2.3.9. For FINITE-^uCHOICE-l, we have that the optimal value of the tradeoff problem 
Q\i{cc,k) is ((Cc,fc - c(A))log ( c^^-c(A) ))’ 3 sequence Cc,k = C(jk) for the sequence 

of policies 7 ^ in Lemma 2.3.8| 


Proof. Consider the sequence Cc,fe = Vk + c(A). Let us choose a sequence ey = 14 log 
which decreases to zero as I 4 i 0. Let jk £ ffa,M be a sequence of ey-optimal policies for 
FINITE-/xCFIOICE such that Q^jk) < Q*Micc,k) + ey. Then, applying Corollary 


2.3.5 


we have that 


jrtrx ~ Qilk) = O i^{cc^k — c{X)) log fc-c(A) ) ) ■ Then, as 14 i 0, there exists some constant ci 


such that 


Qilk) > —^ -ci((cc^k - c(A))log ( 

Pu- X V \Oc,k - c(A) 


Since Q( 7 fc) < Q*M{cc,k) + ^v, we have that 

A 


Q*M{cc,k) + ey > — ^ -Ci( (Cc,fc - c(A)) log (- 737 ) ) 

Pu-X V \Cc,k-C[X)JJ 


Then we have that 


Q M^Oc,k') ^ 


A 


Pu X 


- (ci -F 1) (Cc,fc - c(A))log 


Oc,k c(^) 


We have that Q\[{cc^k) < Qilk), where -jk is the sequence of policies constructed in Lemma 
Therefore Q^(cc,fc) = Qiil) = ^ - 0 ((cc,fc - c(A)) log ( ,^^^e(A) )) 


2.3.8 


□ 


Remark 2.3.10. We note that the asymptotic characterization of Q*icc) has been obtained only 
for a particular sequence I 4 = Ci'jk) — ciX), where 7 ^ is as in Lemma 


2.3.8 


The set of average 

service cost values that can be achieved depends upon the set of service rates, {0, pi,..., px}, 
available for control. For example, if the set of service rates available for control is {0, Pu, Pk}, 
then the average service cost always corresponds to the set of values I 4 in Proposition 2.3.9[ In 
fact, the asymptotic 0 characterization of Q%jicc) can be obtained for any sequence of Cc,fc such 
that there exists a sequence of non-idling admissible 7 ^ such that ( 7 ( 7 ^) — c(A) = 0 (cc,a: — c(A)). 

Remark 2.3.11. In this thesis, any sequence of admissible policies 7 ^, which achieve the asymptotic 
lower bound is called an order-optimal sequence of policies. For example, the sequence of policies 


7 fc in Lemma 2.3.8 is order-optimal. 
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For obtaining bounds on the average queue length for FINITE-/iCHOICE-2 and FINITE-|uCHOICE- 
3, we use a result, presented in Appendix 2.A.1| that uses a quadratic Lyapunov function to obtain 
bounds on the average queue length. 


Lemma 2.3.12. For FINITE-/.tCH0ICE-2, there exists a sequence of non-idling admissible policies 
7 fc, with a sequence 14 | 0 such that Qijk) = O ^log ^ilk) — c(A) = I 4 . 


Proof. Consider a policy 7 defined as follows : 


a(0) = 0, 

fi{q) = fii, for q e {l,...,qki}, 
p{q) = IJ^u, for qe {qki +I,. 


where gfc, = 




, with U > 0. The sequence of policies 7 ^ is obtained by 


choosing U from a sequence Uk that decreases to zero. 


Now we note that for (7 G {1,... ,qki}, 


TT{q) = 7r(0) — 


a)' 


and for q G {qki -F 1,... } 


vr(g) = 7r(0) 

\PlJ \Pu 




7r(0) 


7r(0) 


1 + 




A 


A - /i; 


Pi 




PlJ An -A 


= 1 , 


= 1 . 


(2.24) 


We note that > log/ ^ \ ^1 -F ■ Since ^ > 1 we have that 


Vmz 


AV“>i + F^i 
w; - \ u 
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Substituting the above lower bound, in (2.24), we have that 


7r(0) < 


U 


U 


l-f'U ^ 


+ 1 


y l-^u ^ J 


or, 7r(0) < 


U 

(A 

y {^u j 


(2.25) 


We note that for 7 , ( 7 ( 7 ) = 7 r( 0).0 + tt^{ ki)c{^ii) + Tr^{ku)c{nu)- Also = c(A) + {m - \)m 
and c(uu) = c(A) + (uu — A)m, where m = ^ Then we have that 

C{l) = TTf,{ki){c{X) + {fj-i - x)m) + TTf,{ku){c{X) + {fiu - X)m), 

< c(A) + m{Tr^{ki){fii - A) + 7 r^(fc„)(//„ - A)). 

We note that 7 is admissible, therefore, we have that iT^{ki)^i + TT^{ku)Hu = A. Hence, 

TTf,{ki){fii - A) + 7r^{ku){tiu - A) = 7 r( 0 )A 


Then, 


C{j) < c(A) + m 7 r( 0 )A 


(2.26) 


From (2.25), we have that 


C(7)-c(A) = 0([/). 


Let y = ( 7 ( 7 ) - c(A), then F = 0{U). 

From Proposition 2.A.1| with = q^i + 1 and e = fiu — X, we obtain that 


Qii) < 


{Qki + 1)/^M max 

fiu A 2 (//^ A) 


(2.27) 


Now, for the sequence of policies 7 ^ with Uk i 0, qki = O 

= C>(t7fc), (3(7fc) = 

policies 7 fc such that (5(7fc) = C> (^log and ( 7 ( 7 ^) - c(A) = 


0 °s(i 7 ^))- Hence Q( 7 fc) = 
So there exists a sequence of 
Vfc. □ 


Using the asymptotic lower bound on ( 5 ( 7 ) from Corollary 


2.3.7 


and the asymptotic upper bound 


above, and proceeding as for Proposition 2.3.9 (except that ey is a constant e > 0) we obtain the 
following result. 


Proposition 2.3.13. For FINITE-//CHOICE-2, we have that the optimal value of the tradeoff 
problem Qlj{cc,k) is 0 (^log (^ 773 ^( 1 )))’ ^ sequence Cc,k = f4 + c(A), where Vk = (7(7fc)-c(A) 
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for the sequence of policies 7 ^ in Lemma 2.3.12 


The following asymptotic upper bound for FINITE-/iCHOICE-3, is obtained using a procedure similar 
to that for FINITE-^CHOICE-2 in Lemma 2.3. 12[ 


Lemma 2.3.14. For FINITE-|uCH0ICE-3, there exists a sequence of non-idling admissible policies 
7 fc with a sequence I 4 | 0 such that Qi'jk) = O with C{'yk) — c(A) = I 4 . 

Proof. Consider a policy 7 defined as follows : 

m = 0 , 

p { q ) = A, for g e { 1 ,... ,gA}, 

pIq) = P, qe{qx + l,...}, 

where = min{/Xfc : > A} and q\ is chosen as with U > 0. The sequence of policies jk 

is obtained by choosing U from a sequence Uk i 0. 

We note that for g G {1,..., qx}, as ^{q) = A, we have that Tr{q) = vr(0). And for q G {gA + 1,---}, 
we have that TT{q) = 7r(0) . As = 1’ we have that 


.( 0 ) < < u. 

U fx p — A 


(2.28) 


We note that C'( 7 ) = 7 r( 0).0 -F 7 r^(A)c(A) -F We have that c(//') = c(A) -F m(/i' — A) 

where m = We note that 7 is admissible. Therefore, 7 r( 0).0 -F 7 r^(A)A -F 7 r^(/i')/u' = A, 

or 7 r^(//')(^' — A) = 7 r( 0 )A. Thus 


C{l) = vr^(A)c(A)-F 7 r^(^')(c(A)-Fm(^'- A)), 
= c(A)-F mA 7 r( 0 ) — c(A) 7 r( 0 ), 

< c{X) + {mX — c{X))U. 


(2.29) 


A ^=7, 


Using (2.28) we obtain that, for the policy 7 , C{'y) — c(A) = O [U). Let V = CIj) — c(A). Then 
V = 0{U). 


To use Proposition 2.A.1 we set q^ = qx + 1 and e = /i' — A. We obtain that 


7=)^ N, ^ {QX + ^)p' I A -F rmax 


(2.30) 


For the policy 7 ^, as ^a = O (), we have that Qi'yk) = C* ( 17 :) ■ Since Vk = 0{Uk), we have 


Uk 
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that Qijk) = O Hence there exists a sequence of non-idling admissible policies 7 ^, with 

Qilk) = O (^) and C{jk) - c(A) = Vk. □ 


Using the asymptotic lower bound on Q^'jk) from Corollary 2.3.7 and the asymptotic upper bound 
above, and proceeding as in Proposition 2.3.9| (except that ey is a constant e > 0), we obtain the 
following result. 


Proposition 2.3.15. For FINITE-//CHOICE-3, we have that the optimal value of the tradeoff 
problem Q^(cc,fc) is 0 ■ for a sequence Cc,k = f4 + c(A), where Vk = C{jk) - c(A) for 


the sequence of policies jk in Lemma 2.3.14 


2.3.3 Asymptotic characterization of order optimal admissible policies 


The above approach that characterizes the asymptotic behaviour of the average queue length, via 
an upper bound on the stationary probability distribution of the queue length for admissible policies, 
is used throughout this thesis. In this section, we discuss an advantage that this approach has over 
methods proposed in [7] or We illustrate how the above approach can be used to obtain 

an asymptotic characterization of any sequence of order-optimal admissible policies 7 ^ for which 
C{'fk) i c(A). Such an asymptotic characterization may lead to a reduction in the search space for 


the e-optimal admissible policy for (2.4) as Cc ~ c(A). 


We note that an admissible policy is equivalently described by the sequence (go = 0, gi,..., qk-i, Qk 
00 ). In this section, we discuss how asymptotic bounds on qk can be obtained for an admissible 
policy with C{'y) « c(A). We consider only FINITE-^CHOICE-2 and FINITE-//CHOICE-3 in this 
section, since these are the cases for which the design of policies is more critical (since Q*{cc) 
increases to infinity as Cc I c(A)). 


We note that for special classes of admissible policies, such as buffer-partitioning policies proposed 
in [7], intuition about qk can be obtained from the asymptotic characterization of (5(7) for a 7 such 
that C{'y) ^ c(A). We note that buffer partitioning policies use only a specific set of rates, e.g., for 
FINITE-/iCHOICE-2, only the rates {0,pi,pu} are used. Since pi < X < pu, intuitively, we expect 
that the partition ( 5 ( 7 ), which implies that g^j = XI ^log _e(A) )) ^^7 feasible policy as 

Cc i c(A). We note that the above asymptotic behaviour for the partition can be surmised from the 
asymptotic behaviour of Q{'y), which may be obtained via methods as in [43| or |^. However, in 
the following, we discuss how bounds on the stationary probability distribution of the queue length 
are useful in obtaining a much more refined asymptotic characterization of the policy. 
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Two inequalities: 


In this section, we discuss two inequalities, which form the basis for the asymptotic character¬ 
ization of any sequence of order-optimal policies. Let . Let Pi <Q< 9 ^} and 

Pu { 9 ^ < <5 < ( 7 ^} be any lower bound and upper bound to Pr {( 7 ^ < Q < (?^}, i e.. Pi < Q < < 7 ^} < 
Pr{( 7 ^ <Q < < 7 ^} < Pu{q^ < Q < q^}- Also let 7ri{q) and TTuiq) be any lower bound and up¬ 
per bound to 7r(g), i.e., 7ri{q) < 7:{q) < TTuiq)- The asymptotic characterization of any policy is 
obtained using the following two observations. If qi is the largest integer such that 

P+<ii 

TTuiq) < Pi [q^ < Q < q^}, (2-31) 

q=qT 

then qi < q^ — q^. If qu is the smallest integer such that 

q^+qu 

^ TTiiq) > Pu {q^ < Q < q"^} , (2.32) 

q=qT 

then qi > q^ — q^. We note that bounds on qj^ — qu-i + 1, which is the set of queue lengths for 
which the service rate ij,k is used, can be obtained by using q^ = qu-i + 1 and q^ = q^- 


The bounds - Pi{.}, Pu{-}, vr/(.), and 7r„(.) : 


In the following we discuss how Pi{-}, Pu{-}, '^ii-)< 3 nd TTui-) can be obtained. We note that we 
consider only cases where q^ = qi^/ + 1 and q'^ = q^, where k' < k. Then < Q < q^} = 

Prink'+i < ^^iQ) < ^lk} = ELfc'-ri^MW- 

Consider any admissible policy 7 . Then for any k, we can obtain a lower bound on 7 r^(A:) as the 
optimal value of the linear program: 

min TT^ik) 

K 

such that ^ 7 r^(/c) = l, 

fc=o 
K 

'^TT^ik)^ik = A, 

fc =0 
K 

^TTf,ik) icink) - lifTk)) = Cij) - c(A) 

k=0 

An upper bound on TT^ik) can be obtained by maximising ir^ik) in the above linear program. 
However, we note that finding the above bounds analytically is difficult. Hence, in the following we 
find other bounds on TT^ik) or Yln=k'+ 1 Ti-i^)< which can be expressed analytically. We note that 


(2.33) 

(2.34) 

(2.35) 

(2.36) 
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these analytical bounds are obtained from the constraints in the above linear program. 


We note that for k < ki and k > ku, from Lemma 


2.3.2 


we have that 7r^(A;) < 


Proceeding as in the proof of Lemma 2.3.2 we can also show that if A: < A;; or k' > ku, then 


k 

E 

n=k'+l 


-n-fiik) < 


C(7)-c(A) 


{k’+l,...,k\ (c(/^n) Kl-^n)) 


min. 


(2.37) 


nG 


If ki = ku, as in FINITE-/iCH0ICE-3, then from (2.34p and (|2.36|), we have that 

TT^ikl) = 1 - X] ^ 


C(7)-c(A) 


k^ki 


(c(/Ufc) 


(2.38) 


If ki < ku, as in FINITE-^CHOICE-2, then again from (|2.34p and (|2.36|), we have that 

ku 

TTf,ik) > 1 


C(7) - c(A) 


k=k. 


mmk<ki,k>k,_, {c{fJ.k) - Kk'k))' 


(2.39) 


Suppose we are interested in Ylk=k Then we have that 


+ Y > 1 


k=ki 


k=k+l 


Ch) - c(A) 

(c(/ifc) - Kf^k))' 


(2.40) 


\f k = ku, then we note that the RHS is a lower bound. If < ku, then we proceed as follows 


Y Y 

k=k-\-l 


C'(7) - c(A) 


k=k, 


minfc<fc,,fc>A:„ (c(^fc) - l{fJ-k))' 


(2.41) 


From (2.35), we have that 


k ku 

Y + Y, < A, 

k=ki k=k-\-\ 

k ku 

k=ki k=k+l 


Or, 


Y ^- 

k=k-\-l 
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Therefore, we have that 


'A /.N ^ 1 
2^n^{k) > 1- 


C(7) - c(A) 


k=k, 

k 




Mfc minfc<fc,,fc>fc„ (c(/ifc) - /(/ifc))' 

C(7) - c(A) 


fc=fci 


Hj, - m m.mk<ki,k>ku (c(/Ufc) - lilJ-k))' 

We note that this lower bound is non-negative only if > A. We note that 

Y1 T^t^ik){c{fkk)-l{fJ.k)) = C{'y)-c{X), 

k<ki yk>ku 

Ch) - c(A) 


(2.42) 


T^k^ik) > 


k<ki ,k>ku 


^^^k<ki,k>ku 


(2.43) 


We note that for FINITE-//CH0ICE-2, the best lower bound that can be obtained for Ylk<ki 
and Ylk>ku separately is zero. For example, if we try to obtain a lower bound on Ylk<ki 


since the constraints (2.34), (2.35), and (2.36) can be met by assigning positive probability to 
fJ-k > ki only, we obtain the trivial lower bound that Ylk<ki'^k-i^) — 0- Flowever, for FINITE- 
iuCHOICE-3 we have that 


Y ^k'k - ^) + Y ^k-k - A) = 0, from ( |2.35D , 


k<ki 


k ^ kii 


Y ikk - = J]]vr^(A;)(A-//fc). 

k>ku k<ki 


(2.44) 


To obtain a lower bound on Ylk>ku'^k-(k) we proceed as follows. From (2.44) 


~ A 
A — 


- A) ^ vr^(fc) > (A-Aifc,-i) J]] 7r^(/c), 

k>ku k<ki 

C(7) - c(A) 


Y + Y - 


k>ku 


k ^ kn 


maxfc<fc,,fc>fc„ (c(Mfc) - K^^k)) 


, from (2.43), 


Or, we have that 


Y 


> 


k^ky, 


C(7) - c(A) 


^ f^ki — 1 

f^ki — 1 \^^^k<ki,k>ku i^if^k) ^(/^A:)) 


(2.45) 


To obtain a lower bound on Ylk<k proceed as follows from (2.44) 


^k<ki 

{^J^k■a + l - A)E-^ (k) < A E 

k>ku k<ki 


A 


^J-ku+i - A 


Y + Y - 


C(7) - c(A) 


k<ki 


k<ki 


maxfc<fcj^fc>fc„ (c(Mfc) - KfJ-k)) 


, from (2.43), 
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Or, we have that 


Y1 ^ 

k<ki 


fJ-k^+i - A /_ C( 7 ) - c(A) _ 

/^fcu+l \^^^k<ki,k>ku (.^{fkk') ^(/^A:)) 


(2.46) 


We now consider a method to obtain an upper bound on 
we have 




ki<k<k ’’’m 


i(/c). We note that from (2.34) 


k<ki,k>ku ki<k<k k<k<ku 

k<k<ku 


= 1, 


< 1 - 


C(7)-c(A) 


maXfc<A:,,fc>fc„ (c(/Afc) - KfAfc)) 


, from (2.43) 


(2.47) 


We note that RHS is the upper bound if ki 


follows. From (2.35), we have that 


ku or k = ku- Suppose k < ku, then we proceed as 


^ 7r^(A:)/ifc + ^ Trij,{k)fik + ^ vr^(A:)//fc + ^ vr^(A:)//fc = A, 

ki<k<k k<k<ku ku<k 

l^ki-l fiik) + 7r^(fe)+AAu TTnik) + IJ-K 

ki<k<k k<k<ku ku<k 


Or we have that 


Y 1 - 

k ^ k^^ ku 

Y 1 - 

/c ^ k^^ ku 


l^u 


H 

fj^u 


Y 1 

ki<k<k 


H 

f^U 


Y 1 

ki<k<k 


/Afcj-l 


f^ki-1 


fc<fci ku<k 

C{l) - c(A) _ m 

min„<fc, {c{^ln) - KlJ-n)) k^u 


c{l) - C(A) 

mm„>fc„ {c{kn) - Kkn))' 


From (2.47) we have that 


Y ^ 

ki<k<k 

< 


g(7)-c(A) 

^a^k<ki,k>ku {c{kk) - l{kk)) ’ 

i_A + « '.w + ——+ 

ku ku ^ - ku min„<fc, [c[kn) - Kknj) 

ki<k<k 

kK C(7) - c(A) _ 67(7) - c(A) _ 

ku raiUn^ku {c{kn) - Kkn)) maXfc<fc(,fc>fc„ {c{kk) - l{kk)) ’ 


1 - Y “ 

k<k<ku 
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Or, we have that 


1_ ^ 
I^U 


y TT (fc) < 1 ^ I g( 7 )-c(A) 

^ ^ Hu fj-u miUncki ic{Hn) - KHu)) 


ki<k<k 


t^K 


C( 7 )-c(A) 


+ 

C( 7 ) - c(A) 


Hu niin„>fc„ (c(//„) - l{Hn)) maxfc<fc,,A:>fc„ (c(/ifc) - l{Hk)) ’ 


X] - 

ki<k<k 




+ Hki-l- 


C( 7 )-c(A) 


+ 


Hu-H'k ' ‘ (c(^„) -/(/i„)) 

y7)-c(A) y7)-c(A) 


(2.48) 


Hk- 


minn>fc„ (c(Atn) - ^“maxfc<fc;,fc>fc„ (c(/ifc) - /(/^fc)) ’ 

We note that the above bound is less than one, in the limit as C'( 7 ) J, c(A) only if /x^ < A. 

We note that the above bounds can be used to obtain upper and lower bounds on 
in many cases. We then obtain the lower and upper bounds Pi{.} and using 

Now we discuss how 'Ki{q) and iTuio) can be obtained. 

We note that if +1 and q^ = qk for k' < k, then for any q such that q^ < q < q^, we have 

that 


7r(g) < 7r(gfc') 

vr(^) < 7r{qk) 


A 




Hk'+l 
f^yk-q 

A ) 


or. 


We can then bound 7r{qk') by nyk') or 7r{qk) by 7 r^(/c), which leads to an upper bound TTuiq) for 
q^<q< q^. We note that a similar upper bound on 7r{q) has been used in the asymptotic analysis 
of Q*{cc) above. 

We are only able to obtain asymptotic lower bounds on 7 r( 0 ), for any sequence of non-idling order 
optimal admissible policies 'jk- The asymptotic lower bounds are obtained using the same method as 
in the proof of [44l Theorem 2]. We consider the DTMC which is obtained by uniformizing 


Q{t) at rate r^, as in Appendix 2.A Then, we note that the stationary distribution of Qd[iTi] 
is the same as that of Q{t) under the policy 7 . We proceed as in |443 by assuming that the 
process is stationary at m = 0. Then, from Markov inequality we have that the probability that 
t?d[0] < [ 2 ( 5 ( 7 )] there are no arrivals (or up-transitions for Qd[m]) in k successive slots is at 
least 5 (^1 “ ■ Suppose k = [ 2 ( 5 ( 7 )] + 1- Then we have that the probability that there is no 

service (or down-transition for Qd[m]) in k slots is at least i fl —. Therefore, we have that 


7r(0) > 


2k 


With our choice of k, we have that 


7r(0) > 


1-^ 


^([2Q(7)]+1) 


2([2g(7)] +1) 


(2.49) 
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We note that for the policy under consideration, for FINITE-;uCH0ICE-2 Q{'y) = 0 (log (^)) and 
for FINITE-/xCHOICE-3 Qij) = 0 (^), as F 4 , 0. Therefore for small enough V, we have that 


7r(0) > 


1_^\(2Q(7)+2) 

'f'u J 

4(5(7) 


7r(0) > 7rz(0) 


n > for FINITE-^CHOICE-2, 

^ (1 “ FINITE-^CHOICE-3. 


(2.50) 


We note that an asymptotic lower bound 'Ki{q) can then be obtained since 7r(g) > 'Ki{q) = 
7r;(0) . but this bound is very weak in most cases. 


Asymptotic characterization: 


In this section, we obtain asymptotic bounds on q^. We note that since we are not able to obtain 
analytical forms for Pi {.}, Pu {.}, and 7r„(.) in all cases, we are not able to obtain asymptotic 
bounds on qk,'^k G {1,... ,K}. The asymptotic bounds on qk are obtained using the methodology 
described in Section [2.3.31 


Proposition 2.3.16. For FINITE-//CFIOICE-2, for any sequence of non-idling order-optimal admis¬ 
sible policies 7 fc, with C{'yk) — c(A) = I 4 , we have that 


Pk 


O (^log (^log (4))) , if 1 < < A:; - 1 , 

' ^ ( 1 ^)) ’ 'f ^ H 

^ (log (tr)) ’ 'f ^ is such that > A. 


Proof. Consider any policy 7 in the sequence with I 4 = 14. Consider any k such that I < k < ki — 1. 


From (2.37), we have that 




< 


V 


A V 


k=0 


mm 


™ne{0,...,fc} (o(/^«) - ^(frn)) Cl' 


From (2.32), we have that if qj, ^ is the smallest integer such that 

I 

5=0 




V 

Cl ' 
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then q~^ < Substituting 7r;(g) = 7rz(0) ( ^) , we have that 


Qk,ii 


-SU)' * 






V 

Cl' 


> 


Cl' 


Or we have that q^^ is the smallest integer such that 


q-k^^ > logj^_ 


X- 

1 H- ^ 


V 


H Cl 71^(0) 


Since 7r/(0) = O log^-j- ) J (2-50), we have that q^ < qj,^ = O (log (log (y)))■ We do not 
have any asymptotic lower bounds for q^,! < k < ki — 1. 


Let us now consider k such that ki < k and < A. Then from (2.37) and (2.48), we have that 


k ki — 1 k 

Pr (0 < Q < gj = '^TT^{k) = ^ 7r^{k) + ^ 7r^{k), 

k=0 k=0 k=ki 

V Hu - X 


< 


+ 




V 


minne{ 0 ,...,fci-i} (c(Ain) - /(^n)) ' Hu-Hk ' ‘ " mmn<ki (ciHn) - KHu)) 

V V 


+ 


HK —r 


min„>fc„ (c(Hn) - KHti)) (c(//fc) - KHk)) 

<1 


= Pu{^<Q <qk] ■ 


We also have that Vg < qj^, '^i{q) = 7^'/(0) ■ Then, from (2.32), if ^ is the smallest integer 

such that 

ar. 


^k,u 

Y.Mq)>^^^^^ + oiv), 


then q^ < q^^. Then, we obtain that qku~^ (f))- Therefore, for k such that h < k < X, 
q-^ = 0{log{P)). 


Now consider k such that k < ku and A < /i^. From (2.42), we have that 




> 






k=ki 


H - W Cl 


where ci = m.m.k<ki,k>ku {c{Hk) - Kk-k))- Then 


^vr^(fe) >Pi{0<Q<q-,} = - ^V, 


k=0 


kk - kl Cl 
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For 0 < O' < qj,, we have that 7r(g) < 7r„(g) = 7r„(0) 
integer such that 


'^TTu{q) <Pi{^<Q< q-k] 

q=0 


From (2.31) if qj.^ is the largest 


then ^ Substituting for 7r„(g) and Pi {O < Q < g^j, we have that 


Ik,I 

X^7r„(0) 


q=0 


AV < _ i^v. 

/uiy “/Ufc-w Cl 


Since tTu{0) = 0{V), we have that g^ ^ = O (log (y)) ■ 


□ 


Proposition 2.3.17. For FINITE-//CFIOICE-3, for any sequence of non-idling order-optimal admis¬ 
sible policies 7 fc, with C{'yk) — c(A) = 14, we have that 


Pk = 



0(1), if = ^kkl-l, 



Proof. The methods used in this proof are similar to that used for the proof of Proposition |2.3.16 
We first consider k such that < A. Since 




< 


V 


A V 


k=0 


mm 


ne{0,...,fc} Kpn)) Cl 


from (2.37) and 7r/(0) = 0 (^17 ^1 — Pj ^ j from (2.50), we have that „ = O (y). From (2.46), 
for fc = /c; — 1, we have that 

0<k<ki-l 


Pku+1 VC2 


where C 2 = m.axk<^ki,k>ku (c(Mfc) - KPk))- Then using 7r„(g) = 7r„(0) we have that q-^ ^ = 

0(1), so that g^ = 0(1). 

Consider k such that = A. We note that 7r^(A) < 1, then Tri{q) = Tri{0),\/q < g^. Then we 
obtain that %u — ^^ ^ 
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Ex. Identifier 

A 

c(A) 

Type 

Tradeoff plot 

E-111 

0.25 

0.125 

FINITE-^CHOICE-1 


2.3 


E-112 

0.50 

0.250 

FINITE-/iCHOICE-3 


2.5 


E-113 

0.75 

0.625 

FINITE-^CHOICE-2 


2.7 



Table 2.2: Numerical examples with c(^) = for /i G {0,0.5,!}. 


Ex. Identifier 

A 

c(A) 

Type 

Tradeoff plot 

E-121 

0.25 

0.015625 

FINITE-/iCHOICE-l 


2.4 


E-122 

0.50 

0.031250 

FINITE-/iCHOICE-3 


2.6 


E-123 

0.75 

0.515600 

FINITE-/iCHOICE-2 


2.8 



Table 2.3: Numerical examples with c(^) = f/’ for /i G {0,0.5,!}. 


Now consider any k such that > A. We have that ^ ~ where 

Cl = mmk<k,,k>ki {c(/ifc) - l(fJ-k)}- Since 7r„(g) = 7r„(g'fc,-i + 1), for all q > qki-i, we have that 
Qki,l = ^ {y) ■ n 

2.3.4 Numerical examples 

In this section, we consider some numerical examples for FINITE-/iCHOICE. In the examples, we 
compare the bounds on Q*{cc) which were obtained above, with the optimal tradeoff curve for 
FINITE-/iCHOICE, which is obtained by the numerical solution of an MDP, obtained by uniformiza- 
tion as in |5j. We now state, the chosen parameters for each numerical example and the quantities 
plotted in the corresponding figures. Each numerical example is identified by "E-abc”, where a,b, 
and c are numbers. For each numerical example, we choose the set of service rates and the service 
cost function c(/i). Then we consider a set of arrival rates, for each of which the bounds and 
the optimal tradeoff are plotted. We note that for each value of A, we obtain a corresponding 
minimum average service cost c(A). All the numerical examples that we consider in this section, 
along with their parameters, are given in Tables [2^ |2.3| |2.4[ and |2.5| along with references to their 
corresponding plots. 


Ex. Identifier 

A 

c(A) 

Type 

Tradeoff plot 

E-211 

0.10 

0.02 

FINITE-//CHOICE-1 


2.9 


E-212 

0.20 

0.04 

FINITE-/iCHOICE-3 


2.11 


E-213 

0.25 

0.07 

FINITE-//CHOICE-2 


2.13 


E-214 

0.70 

0.50 

FINITE-/iCHOICE-2 


2.15 



Table 2.4: Numerical examples with c(/i) = fj? for ^ G {0,0.2, 0.4,0.5,0.6,0.8, !}. 
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Ex. Identifier 

A 

c(A) 

Type 

Tradeoff plot 

E-221 

0.10 

0.00016 

FINITE-gCHOICE-1 


2.10 


E-222 

0.20 

0.00032 

FINITE-gCHOICE-3 


2.12 


E-223 

0.25 

0.00280 

FINITE-gCHOICE-2 


2.14 


E-224 

0.70 

0.20272 

FINITE-gCHOICE-2 


2.16 



Table 2.5: Numerical examples with c(^) = /i® for n G {0,0.2, 0.4,0.5, 0.6,0.8,1}. 


The Optimal set of points in each plot is obtained by considering a MDP with the single stage cost 
given by (7 + /3c{p), and solving for the optimal infinite horizon average cost policy. The state of 
the MDP corresponds to the queue length q, and the set of actions taken at each state q is the set 
of service rates. Here /? is a positive Lagrange multiplier. The MDP is obtained by uniformization 
at rate = A + rmax- The transitions in the uniformized MDP are as follows : a) for q = 0, the 
Markov chain moves to g = 1 with probability —, and stays in g = 0 with probability 1 —and 


b) for g > 0, the Markov chain moves to g + 1 with probability —, to g — 1 with probability 


mM 


and stays in g with probability 1 — 




. We note that the state space of the MDP is truncated 


at a maximum queue length, which is such that the optimal value does not change appreciably with 
further increase in this maximum queue length. The Optimal points are obtained by varying /3. The 
x-coordinate of a point corresponding to a value of /3 is the difference between the average service 
cost, for the /3-optimal policy for the MDP, and c(A) while the g-coordinate is the average queue 


length for the /3-optimal policy. The Lower bound (Analytical) curve in each plot is: a) (2.11) 
for FINITE-/iCHOICE-l, b) ( [IT^ for FINITE-/iCHOICE-2, and c) ( [2l^ for FINITE-/xCHOICE-3. 
The Upper bound (Analytical) curve in each plot is obtained as follows: a) for FINITE-/iCHOICE-l, 
we choose the sequence gfc„ to be a sequence of increasing positive integers, and for each q^^ obtain 
the bound on the average queue length from (2.20), and the average service cost from ( 2.19D , b) 
for FINITE-gCHOICE-2, we choose the sequence Uk to be a decreasing sequence, and for each Uk 


obtain the bound on the average queue length from (2.27) and the average service cost from (2.26), 
c) for FINITE-/iCHOICE-3, we choose the sequence Uk to be a decreasing sequence, and for each 


Uk obtain the bound on the average queue length and the average service cost from (2.30) and 


(2.29) respectively. We note that the bounds on the average queue length which are obtained using 


the Lyapunov drift method in Proposition 2.A.1 are usually weak (although they give the correct 
order behaviour). Therefore, for the sequence of policies which we have considered for Upper bound 
(Analytical), we also evaluate the actual average service cost and average queue length, by obtaining 
the stationary probability of the queue length for the system with truncated state space. This curve 
is denoted as Upper bound in the plots. We note that discrete points are obtained while varying /3, 
time sharing of the policies corresponding to these points leads to the continuous curves shown in 
the figures. 


We note that in all the cases that we have considered, both the analytical upper and lower bounds 
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Figure 2.3: Q*(cc) as a function of V, where Cc = c(A) + V, for A = 0.25 and c(^) = for /j, G {0,0.5,1}. 



Figure 2.4: Q*{cc) as a function of V, where Cc = c(A) + V, for A = 0.25 and c(^) = /i® for /i G {0,0.5,1}. 
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Figure 2.5: Q*{cc) as a function of V, where Cc = c(A) + V, for A = 0.50 and c(/i) = for /x S {0, 0.5,1}. 



Figure 2.6: Q*{cc) as a function of V, where Cc = c(A) + V, for A = 0.50 and c(/x) = fj,^ for fj, € {0, 0.5,1}. 
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Figure 2.7: Q*(cc) as a function of V, where Cc = c(A) + V, for A = 0.75 and c(^) = for /j, G {0,0.5,1}. 



Figure 2.8: Q*{cc) as a function of V, where Cc = c(A) + V, for A = 0.75 and c(^) = /i® for /i G {0,0.5,1}. 
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Figure 2.9: Q*{Cc) as a function of V, where Cc 


c(A) + V, for A 


0.10 and c{fi) = 


{0,0.2,0.4,0.5,0.6,0.8,1}. 


for /X G 



Figure 2.10: Q*{cc) as a function of V, where Cc = c(A) + V, for A = 0.10 and c(/x) = for ^ G 
10,0.2,0.4,0.5,0.6,0.8,1}. 
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Figure 2.11: Q*{cc) as a function of V, where Cc = c(A) + V, for A = 0.20 and c(/i) = /i^ for fj, G 
{0,0.2,0.4,0.5,0.6,0.8,1}. 



Figure 2.12: Q*(Cc) as a function of V, where Cc = c(A) + V, for A = 0.20 and c(/i) = /z® for /j, G 
10,0.2,0.4,0.5,0.6,0.8,1}. 
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Figure 2.13: Q*{cc) as a function of V, where Cc 


c(A) + V, for A 


0.25 and c(fj.) = 


{0,0.2,0.4,0.5,0.6,0.8,1}. 


for /i e 



Figure 2.14: Q*{cc) as a function of V, where Cc 


c(A) + V, for A = 0.25 and c{fj,) = 


10,0.2,0.4,0.5,0.6,0.8,1}. 


for fi G 
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Figure 2.15: Q*{cc) as a function of V, where Cc = c(A) + V, for A = 0.70 and c(/i) = /i^ for fj, G 
{0,0.2,0.4,0.5,0.6,0.8,1}. 



Figure 2.16: Q*(Cc) as a function of V, where Cc = c(A) + V, for A = 0.70 and c(/i) = /z® for /j, G 
10,0.2,0.4,0.5,0.6,0.8,1}. 
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Figure 2.17: The optimal tradeoff curve for the system with = {0,0.2, 0.4, 0.5, 0.6, 0.8,1}, c(/i) = 
G S, and A = 0.39,0.40, and 0.41. The minimum average service cost rates are c(0.39) = 
0.154, c(0.40) = 0.160, and c(0.41) = 0.169 


are very loose. 


The difference in the asymptotic behaviour of Q*{cc) for FINITE-/xCHOICE-2 and FINITE-/iCHOICE- 
3 is illustrated by the following example. Consider the following example : we choose the set of 
service rates = {0,0.2, 0.4, 0.5,0.6,0.8,1}, and c(/i) = G S. In Figure 


2.17 


we plot the 

tradeoff curve, numerically obtained from a suitably truncated MDP, for A = 0.39,0.40 and 0.41. 
The minimum average service cost rates corresponding to A = 0.39, 0.40 and 0.41 are 0.154, 0.160, 
and 0.169. We note that the difference between the average service cost and c(A) increases when 
A is changed from 0.39 to 0.40 and then decreases when A is increased, since at A = 0.40, the 
average queue length increases at the rate y. 


In Figures 2.18 and 2.19 we illustrate the stationary probability mass functions 7r{q) for optimal 
policies for FINITE-/iCHOICE-2 and FINITE-;uCHOICE-3 respectively. 


2.3.5 An Application 


In this section we discuss the application of the above asymptotic results to the example considered 


in Section 2.1.3 We obtain a sequence of optimal policies for both the M/M/l-PS (I) and the 
M/M/l-PS (II) models which trade off the average transmitter power with the average number of 
flows. Both of these sequences are obtained by the numerical evaluation of the optimal policy for 
a suitably truncated MDP, with single stage cost q + /3P{m) and with state transitions as shown 


in Figure 2.1 for a sequence of /3 > 0. The optimal tradeoff for both M/M/l-PS (I) and (II) 


models are shown in Figure [Z20| We then obtain via simulation, the average power and average 
number of flows for the discrete time system, for the policies obtained from the M/M/l-PS (I) 
and M/M/l-PS (II) models, to obtain a possible (sub-optimal) tradeoff curve as shown in Figure 
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Queue length (q) 

Figure 2.18: Stationary probability mass functions for optimal policies for the system with = 
{0,0.2,0.4,0.5,0.6,0.8,1}, c(/i) = G S, and A = 0.39 (corresponding to FINITE-^CHOICE-2). 

The (log (^)) asymptotic lower bound arises due to the geometrically increasing and decreasing nature of 
TT{q) as I 0. 



Queue length (q) 


Figure 2.19: Stationary probability mass functions for optimal policies for the system with X^ = 
{0,0.2,0.4,0.5,0.6,0.8,1}, c(/z) = G S, and A = 0.40 (corresponding to FINITE-^CHOICE-3). 

The n (^) asymptotic lower bound arises due to the constant nature of 7r(g) as fG Q. 
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Figure 2.20: Tradeoff of average number of flows in the downlink scheduler queue with the average transmitter 


power for the motivating problem considered in Section 2.1.3 for a flow arrival rate 0.2 flows/sec 


2.20 We note that the sequence of policies suggested by both (I) and (II) models have similar 
performance. For this example, the tradeoff curve obtained from the simulation of the policies 
suggested by the analysis, demonstrates that with a IW increase in transmitter power, a 36 fold 
decrease in latency, i.e., from 3 mins to 5 secs, is possible. We note that P^{m) < P‘^{m) for 
m G {1.25,1.50,1.75,2}, i.e., for //>//; given that A = 0.2 (we recall that actual service rates are 
1^). Then for each of the optimal policies for the M/M/l-PS (I) (or (II)) model, for which the 
average number of flows is large, P^ {m{Q)) < P{m{Q)) < P‘^{m{Q)) with high probability. Thus, 
the tradeoff curves obtained from the M/M/l-PS (I) and (II) models are approximate upper and 
lower bounds to tradeoff curve for the discrete time system, especially when the average number 
of flows are large. Then the asymptotic analysis in the above sections, leads to asymptotic upper 
and lower bounds on the average transmitter power as a function of the average number of flows 
for the discrete time system. Let c^{m) and c^{m) be the lower convex envelopes of P^{m) and 
P‘^{m) respectively. Since fj,i = 1.25/6.4 < A = 0.2 < = 1.50/6.4, by considering the sequence 
of optimal policies from the M/M/l-PS (II) model, the average transmitter power for the discrete 
time system can bounded below by c^(6.4A) 4-11 (e~^) and bounded above by c^(6.4A) + 0 (e“^), 
for large q, where q is the average number of flows. 


2.4 Conclusions 

The main purpose of this chapter is the illustration of the techniques involved in: (i) the derivation 
of asymptotic bounds on Q*{cc) and Q*j^{cc) for admissible policies and (ii) asymptotic bounds on 
order-optimal admissible policies in the regime JR, using a simple queueing model. 

For FINITE-//CHOICE, we observe that the constraint on the average service cost leads to a re¬ 
striction on the stationary probability of service rates which in turn restricts the behaviour of the 
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stationary probability of the queue length in the asymptotic regime 5R, where V = Cc — c(A) | 0. 
For FINITE-/iCHOICE-2 and FINITE-//CHOICE-3, we note that the stationary probability of service 
rates less than fj,i or greater than goes to zero as f/ I 0. Since, state 0 uses a service rate of 0, 
the above fact implies that 7r(0) | 0 as F | 0. Hence, it is intuitive that the average queue length 
has to increase. For FINITE-|uCHOICE-3, as /i; = //« = A, as f/ | 0, the stationary probability 
that any service rate other than A is used approaches zero. Then we observe that the stationary 
probability 7r(g) becomes equal and is 0{V) for all q which occur with high probability, as y | 0, 
since 7r(g)A = 7r{q + l)/j,{q),q > 0 and /x(g) = A for all q occurring with high probability. The above 
constant nature of ir{q) leads to the Q (^) asymptotic lower bound. For FINITE-/iCHOICE-2, as 
y I 0, service rates < \ and fiu > ^ could be used. Therefore, intuitively, one expects that the 
rate at which Q*(cc) increases is less for FINITE-/iCHOICE-2 compared to FINITE-/iCHOICE-3. 
Furthermore, since vr(g)A = 7r(g + l)//(g),g > 0, we observe that the stationary probability TT{q) 
has a geometric growth and decay with growth and decay rates at most and at least respec- 
tively, for the set of queue lengths occurring with high probability, as y | 0. The above geometric 
growth and decay of 7r(g) leads to the Q. (log (’^)) asymptotic lower bound in Lemma 


2.3.6 


The 


above behaviour of the stationary probability in the different cases motivates us to analyse the 
tradeoff problem for the discrete time queue in Chapter 4, by constructing bounds on the stationary 
probability of the queue length, which have the same behaviour in the respective cases. 


We note that the sequence of policies which achieve the asymptotic order behaviour in Section 2.3.2 
for FINITE-/iCHOICE-2 and FINITE-;uCHOICE-3 are similar to buffer partitioning policies. We note 
that the buffer partitions that were used for these sequences of policies scaled as Vl (log (^)) and 
n(^) for FINITE-/xCHOICE-2 and FINITE-/xCHOICE-3 respectively. This scaling of the buffer 
partitions was suggested by the asymptotic lower bounds that were derived in Section 2.3. 1| and we 
shall see that similar ideas can be used in the design of buffer partitioning policies for discrete time 
systems. Furthermore, we have also derived asymptotic bounds on any sequence of order-optimal 


policies in Section 2.3.3 


We note that FINITE-^CHOICE-1, where the average queue length increases only to a finite value, 
even when the average service cost rate is the minimum possible c(A), has been hitherto unidentified 
in the literature. To the best of our knowledge, the asymptotic characterizations of the optimal 
tradeoff curve obtained in this chapter, for all the three cases, were previously not known for the 
state dependent M/M/1 model. 

The development of the asymptotic results in this chapter, partly motivates the definition of ad¬ 
missible policies for the discrete time queueing model in Chapter 4. The non-idling nature of the 
optimal policy which has been obtained in Lemma 2.2.5[ motivates us to consider whether the 
optimal policy for the discrete time queueing model has the same property also. In Chapter 4, we 
shall show that in fact it does. The non-idling property of the optimal policy and its relation with 
the place-holder bit scheduling policies [45] are discussed in more detail in Chapter 4. We have also 


68 











illustrated the utility of the simple state dependent M/M/1 queueing model in the analytical study 
of scheduling schemes for next generation wireless systems using the example in Section [2.1.3 


69 





Appendix 


2.A Uniformization and a bound on the average queue length 


Let Vu = rmax -\-ra,max- Consider a discrete time Markov chain {Qdi'm]) which is obtained by uni¬ 
formization of {Q{t)) at rate Vu- The transition probabilities, Pqi,q 2 = P{Qd['>TT' + 1] = (l 2 \Qd['m] = 
gi), m G Z_|_, of the DTMC are as follows : 


Po,o = 1 - 


A(0) 


r,. 


A(0) 

Po,i = -, 

Vu 

_ K(l) vy ^ 1 

Pq,q-\-l — ; ^ 1 ; 

‘^u 

_ Kq) 

Pq,q-^ — ’ 

' U 


Pq,q = 1 - 


Kq) + Kq) 


We note that for an admissible policy 7, the stationary distribution vr is the same for both the CTMC 
Q{t) and the DTMC Qdi'm], Thus 'E-^Qd = Qil), ETru{X{Qd)) = and E.,^c{p{Qd)) = ^( 7 ). 
The following proposition states an upper bound on < 5 ( 7 ) for an admissible policy 7 subject to an 
assumption about the structure of the policy. 


Proposition 2.A.I. Assume that the admissible policy 7 is such that there exists a such that 
p{qe) — X{qe) > e, for some e > 0. Then 

Q(7) < + ^ (2.51) 

e 2e 


Proof. Let L{q) = g^. We use L(g) as a Lyapunov function to derive the above upper bound. The 
expected Lyapunov drift A(g) = 

E [L{Qd[m + 1]) - L{Qd[m])\Qd[m] = q] (2-52) 

We have that 

ru Tu 
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Note that from the admissibility of 7 , Vg > q^, ^(q) — X[q) > e. So for q> q^ we have that 

Ar,) < + 

Tu ru 


For q < q^, 


AW, ^ + 2^ _ %w) - Afa)) + itiliM, 


' u 


For q < q^, fj,{q) — X{q) < e. Therefore 


A(,) <=^ + %- „(,) + A{,)) + 

rry> /y* 

u ' u ' u 

-2qe 

^-1-(e + fa,max) + 1- 

fu fu 


For all q, we therefore have that 


, , , —2qe 2qe, 

A(g) <-1-(e + fa,max) + 1- 

fu fu 


Hence from [351 Theorem A.4.3] we have that 


(253) 


«7) = KQi < 

e 2e 


□ 
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CHAPTER 3 


On the tradeoff of average queue length, average service cost, and 
average utility for the state dependent M/M/1 queue: Part II 


3.1 Introduction 


We continue our analysis of the tradeoff problem for the state dependent M/M/1 model in this 
chapter. We note that FINITE-/iCHOICE was primarily motivated by the wireless network problem 


in Section 2.1.3 for which the set of possible service rates was a finite discrete set. In this chapter, 
we consider the INTERVAL-//CHOICE and INTERVAL-A/xCHOICE problems, which we study with 
the objective of understanding the tradeoff problem for the discrete time queueing model. We note 
that in this chapter, and afe chosen to be finite intervals. 


The method of analysis for INTERVAL-//CHOICE and INTERVAL-A/xCHOICE is similar to that in 
Chapter 2. We again obtain bounds on the stationary probability distribution of the queue length 
for admissible policies, leading to an asymptotic characterization of the average queue length as 
well as order-optimal admissible policies, in the asymptotic regime 5R. We note that the analysis 
for INTERVAL-/iCHOICE can be used to obtain the results for FINITE-//CHOICE. However, in 
Chapter 4, we will see that some of the steps used in the analysis for INTERVAL-/iCHOICE and 
INTERVAL-A^CHOICE, which are different from that for FINITE-//CHOICE, are essential in the 
analysis of the discrete time models. We recall that the consideration of the stationary probability 
distribution of the queue length in the asymptotic regime 5R, for admissible policies (which are 
monotone), as a method for understanding the asymptotic behaviour of the average queue length, 
underlies most of the results obtained in this thesis. We note that in this chapter, since we consider 
queueing models with arrival rate control as well as other forms of c(//) (other than the piecewise 
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linear form in Chapter 2), we are able to obtain new insights in this direction. 

The question arises as to how insights about the asymptotic behaviour of the minimum average 
queue length in the regime JR for the discrete time model can be obtained from the state dependent 
M/M/1 model. It is clear that all the model features for the discrete time model, even with a single 
environment state, cannot be captured by the state dependent M/M/1 model. The model features 
for the state dependent M/M/1 model are the sets T/j and and the functions c(fj,) and u(A), 
with the restriction that fj,(q) and A(q) are deterministic functions of the queue length. For the 
discrete time model with a single environment state, we note that in addition to the sets of possible 
service batch sizes and possible admitted arrival batch sizes, and the cost and utility functions, we 
also have that the service batch size and the amount of arrivals admitted in a slot are randomized 
functions of the history of the process, as discussed in Chapter 1. 

Suppose we consider only the set of stationary policies for the discrete time model, which choose 
the service batch size S (or the amount A of arrivals admitted) as a randomized function of the 
current queue length q only, say with probability distribution (or distributed as Pa\q,r which is a 
function of the current queue length and number of actual arrivals r). Even then, we have to reduce 
Pg^q (or Pa\q^r) 1° ^ value, which can then be modelled by fi{q) (or X{q)). In the following, we 
take as the quantity which represents Pg^q. Thus fi{q) is assumed to correspond to IEp„|q<S' 

and therefore is the set of all values that can take. A similar assumption is made for 

X{q). 

We note that for the discrete time model, the service cost incurred in a slot is a random variable, 
since the batch size S itself is random. At a queue length q, the expected service cost is Ep^|^c(S'). 
Since we have already chosen ij,{q) to correspond to Ep^i^S", a possibility is to choose c(/i) to 
correspond to c(Ep^|_^S). A similar assumption is made for u{X). In retrospect, this turns out to 
be a good choice of model features for the state dependent M/M/1 model (e.g., we obtain the 
asymptotic Berry Gallager lower bound for the M/M/1 model which, with the above choice of 
model features, corresponds to the discrete time model in [7] with admissible policies). We can also 
then surmise that one of the reasons for the asymptotic behaviour of the stationary queue length 
for admissible policies in the regime JR for the discrete time model are the behaviours of asymptotic 
probability distributions of Ep^|_^S'((5) and Kp^^^^A{Q,R), in the regime JR, since for the M/M/1 
model the behaviours of these quantities are significant. 


3.1.1 Overview 


We present the analysis of INTERVAL-/xCHOICE in Section 3.2 The motivating discrete time 
problem for INTERVAL-/iCHOICE is discussed in the same section. We recall that in defining 
INTERVAL-/iCHOICE we considered admissible policies for which X{q) is a constant A for all 


q G Z+. We obtain asymptotic lower and upper bounds on the tradeoff problem in Sections 3.2.1 
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and 3.2.2| respectively. Asymptotic bounds on order-optimal policies for INTERVAL-/iCHOICE, are 
presented in Section 3.2.3 We consider the counterpart INTERVAL-ACHOICE of INTERVAL- 
/iCHOICE, for which ^{q) is constant over all q G Z+ and X{q) is the control variable, in Section 
|3.2.4 The analysis of INTERVAL-A/iCHOICE in presented in Sectionalong with the motivating 
discrete time problem. We present the main conclusions in Section where we discuss the main 
ideas obtained in the asymptotic analysis of INTERVAL-//CHOICE, INTERVAL-ACHOICE, and 
INTERVAL-A/iCHOICE, and how these ideas can be applied to the discrete time model in Chapters 
4, 5, and 6 . 


3.2 Analysis of INTERVAL /iCHOICE 


We recall that for INTERVAL-ACHOICE we restrict to admissible policies 7 such that X{q) = A 
and a(<?) £ [0,rmax], Vg G Z_|_. The tradeoff problem for INTERVAL-aCHOICE is 

minimize Qix) 

such that C{'y) < Cc, 


whose optimal value is denoted as Q*{cc). We also note that a tradeoff problem can be defined 
where we minimize over the set of policies, which includes finite mixtures of pure policies in T^. The 
optimal value of this problem is denoted as Q\j{cc)- Then, as for the case of FINITE-aCHOICE 
(e.g. as in Corollary |2. 3.5 ), asymptotic lower bounds for Q*{cc) can be used to obtain asymptotic 
lower bounds for Q 


M\ 


In fact, the asymptotic lower bounds in the regime JR are the same for 
Q*{cc) and Q%[{cc)- Furthermore, the asymptotic upper bounds that we derive for Q*{cc) are by 
definition upper bounds for Q^(cc). Hence, in the following, we present the results for Q*{cc) only. 

The study of INTERVAL-aCHOICE is classified into: 


INTERVAL-aCHOICE-1 : c(a) is strictly convex for a £ [0,rmax], and 

INTERVAL-aCHOICE-2 : c(a) is piecewise linear. That is, (a) there exists a minimal partition 
of [0,rmax] into intervals {[ai,bi],i G P}} with ai = 0, bp = rmax, and 5* = aj+i, 

and (b) there are linear functions ft such that Va G [a*, 6 *],/^(a) = c(a). 

Remark 3.2.1. We first discuss the motivation for INTERVAL-aCHOICE-1. INTERVAL-aCHOICE- 
1 corresponds to the tradeoff problem for the following discrete time queueing model. Work arrives 
in a batch, of random size, in every slot, into an infinite buffer queue. The state of the queue is the 
amount of unfinished work. We note that for the discrete time model, the amount of unfinished 
work or the queue state evolves on the set of non-negative real numbers. This is approximated by an 
integer-valued queue evolution process in INTERVAL-aCHOICE- 1. The amount of work completed 
in each slot, or the service batch size, can be chosen as a function (possibly randomized), of the 
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current backlog of unfinished work. The choice of the amount of work completed in each slot as a 
function of the current backlog for the discrete time queue, is modelled by the control of the service 
rate, fi{q), in INTERVAL-//CHOICE-1. Hence, the service batch size and the average service batch 
size (or rate) also takes values in the set of non-negative real numbers. As discussed in Section 


3.1 we then assume that ^{q) takes values in an interval, which for technical reasons is assumed 


to be finite. We assume that there is no admission control in the discrete time model, therefore 
we assume that the arrival rate is A for every q for INTERVAL-/xCHOICE-l. For the discrete time 
queue, we assume that there is a service cost associated with the amount of work done in each slot. 
This is modelled by the service cost rate function c(.) in INTERVAL-^CHOICE-1. We note that 
since the amount of work done in a slot can be any real value, the service cost for the discrete time 
model could be a strictly convex function defined on an interval, which provides the motivation for 
assuming c(//) to be strictly convex for INTERVAL-^CHOICE-1. We note that this discrete time 
model is similar to the model considered by Berry and Gallager [7], but with a single fade state. 


The motivating discrete time queueing model for INTERVAL-//CHOICE-2 is very similar to the 
discrete time model discussed above, except that the queue evolution is assumed to be on integers. 
But for stationary randomized policies, the average service batch size could still take any real value 
in a finite interval, and in light of the discussion in Section [3^ ^{q) is again assumed to take any 
value from a finite interval. In Chapter 4, we shall see that then c(Ep^|^S') is piecewise linear, which 
is the motivation for the piecewise linear assumption on c(//) for INTERVAL-|uCHOICE-2. 


For any admissible policy 7, from Jensen's inequality, we have that C{'y) > c(A). We study 
INTERVAL-/iCHOICE-l and INTERVAL-/iCHOICE-2 in the asymptotic regime 5? where the service 
cost constraint Cc approaches c(A), since it can be shown that c(A) = inf.ygra C{'y). 

Similar to FINITE-/iCHOICE, since the asymptotic behaviour of Q*(cc) for INTERVAL-/iCHOICE-2 
depends on the behaviour of c{^) in a neighbourhood of // = A, we consider the following cases for 
INTERVAL-//CHOICE-2: 


INTERVAL-/iCHOICE-2-l : A G (0,6i = hx), 

INTERVAL-/iCHOICE-2-2 : A G (oj = ca, hi = h\) for some i G {2,..., P}, and, 
INTERVAL-;uCHOICE-2-3 : \ = at = a\ for some f G {2,..., P}. 


The motivation for classifying INTERVAL-^CHOICE-2 into the three cases is the same as that for 
FINITE-/iCHOICE. The different cases are illustrated in Figure [3T| We note that for INTERVAL- 
iuCHOICE-l, the function c(/i) is strictly convex for every /i G [0,rmax] and therefore it has no 
subcases. We note that the analysis of INTERVAL-/xCH0ICE-2 is similar to that of FINITE- 
//CHOICE except that now the set of service rates is not a given finite set. We now present the 
asymptotic lower bounds for Q*{cc) in the regime 3^ for the above cases. 
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INTERVAL-//CHOICE-1 INTERVAL-/j,CHOICE-2-l 



Figure 3.1: Illustration of the relationship between A, ni, and along with the minimum average cost c(A) 
and the line lip) for the four cases of the INTERVAL-/iCHOICE problem. 

3.2.1 Asymptotic lower bounds 

Similar to the definition used for FINITE-/iCHOICE, line l{fi) in the case of INTERVAL-/xCHOICE is 
defined as: (a) The tangent to the c{p) curve at A for INTERVAL-//CHOICE-1, (b) the line passing 
through (0,0) and (6i, c(6i)) for INTERVAL-//CHOICE-2-1, (c) the line passing through {a\, c{ax)) 
and {b\,c{bx)) for INTERVAL-/xCHOICE-2-2, and (d) any line that passes through (A,c(A)) with 
a slope m, such that |^=a < m < |^=a (the left and right derivatives respectively) for 

INTERVAL-/tCHOICE-2-3. 

We note that like in the case of FINITE-/iCHOICE, here we find an asymptotic lower bound on 
( 5 ( 7 ) by (a) obtaining an upper bound on the stationary probability for a certain set of service rates 
in terms of ( 7 ( 7 ) and c(A), (b) relating the stationary probability of this set of service rates to the 
stationary probability vr((?), of a set of queue lengths, and (c) obtaining a lower bound on < 3 ( 7 ) in 
terms of 7r(g). 

We first consider INTERVAL-^uCHOICE-l, for which cin) is a strictly convex function of /x G 
[0,rmax]- We make the following assumption regarding c(/x) at /U = A. 

C2 : For INTERVAL-^CHOICE-1, the second derivative of c(/x) is non-zero at /x = A. 

The above assumption has been used in |7). We note that since c(/x) is strictly convex, the second 
derivative of c(/x) is non-zero for all /x G [0,rmax] except for /x in a countable set. We note that 
even though /x(g) G [0, Vmax], the set {/x(g), q G Z+} is only countable. Let {/xq = 0,..., pk, ■ ■ •} 
denote the set of service rates that is used by a policy 7. 
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Lemma 3.2.2. For INTERVAL-|uCFIOICE-l, for any sequence of non-idling admissible policies jk 
such that C{^k) — c(A) = f4 0 we have that Qi'jk) = ^ (^It) ■ 


Proof. Consider a particular policy 7 in the sequence 7 ^ with Vk = V. Let /i* = A — ey, where 
ey > 0 is a function of V. The functional form of ey will be chosen later. We have that 


k=0 q=0 


c(A) -F 


9=0 


d/i 


{p{q) - A) - l{^i{q)) -F G(/x(g) - A) Tr{q), 


{i=X 


where G{x) is a strictly convex function in x as in [71 Proposition 4.2]. We note that c(A) -F 
Pi=\{p{q) - A) = l{n{q)). Thus we have that V = J2^oG(fk(q) - A) 7 r(q). As G(x) is 


dc{fi) 
d/i 

strictly convex in x and fi{q) — A is bounded, we have from Proposition 3.A.1 that G{fi{q) — A) > 
ai{fj,{q) — Xy for some constant ai > 0. Thus we have that 


V > aiY^(fi(q) - Xf7r(q). 

q=0 


(3.1) 


Define = inf : /x(q) > fi*}. From the non-decreasing property of fj,(q) for 7 , we have that 
Pr {fi(Q) < fj.*} = Pr{Q < g^*}- Then, 




V 


Pr{fx(Q) ^ 7 r(g) < , 


(3.2) 


where we have used the upper bound p.ll). We choose ey as 02 '/^, so that a = 

■-■ ^ 1 ^ 0.2 


We choose 02 such that a < 1. In fact, we note that a can be made arbitrarily close to zero by 
the choice of 02 . Therefore, Pr{ijL{Q) > //*} > 1 — §, which can be made arbitrarily close to one. 


We note that for any g < g^*, 7r(g) < Therefore TT{q^*) < Ti{q^* — 1)^ < In order to 
obtain a lower bound on ( 5 ( 7 ), we intend to find the largest g such that Pr {Q < g} < ^. But we 


note that Pr {Q < g^*} < f. Therefore the largest g satisfies 

q , 

7r(g) -F Y ^^( 5 ) ^ T 

q=0 

If g^ satisfies 


<l=<lu* 


9i 


E , . 1 cr 

7r(g) < ^ - 




2’ 
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then qi < q, for a sufficiently small. As Qif) is a birth-death process, we have that 7 r(g)A = 
7 r ((7 -|- -|- 1). Furthermore, if g > q^* we have that ^{q — 1)A > 7r{q)^*. By induction, we 

obtain that for q £ {Qfj.*, ■ • • } 


xy-i. 


A’ 


7r(g) < 7r(g^*) — < 7r(g^. “ 1) “c 


^ \ Q—Q/i* -l“l 


A’ 


/ > xm 


and for any g > 9 /.*, vr(g) < 7r(g^* - 1) ^ 

q=qfj,* m=l 


(3.3) 

(3.4) 


Using the above upper bound on Yll=q , obtain a lower bound ^2 to Qi- If 92 the largest 

integer such that 


TT{q^, 


'?2-9/j*+1 / \ x m 

1) E 

m=l 



a 
2 ’ 


(3.5) 


then EgLo^( 9 ) < 5 and > q^. 


Now we obtain an upper bound on TT{q^* — 1), which is tighter than the upper bound j derived 
before. From ( |3.lD we have that 

T/ °° 

— > - A)V(9) > 

t q=o q<q^j,* 

= (A(9)-A)V(g) + 0 Y 

q<q^i* q>qfi* 

I V 

— Y1 (^( 9 ) “ A)7r(g) I (using Jensen's inequality as in j7]). (3.6) 

But, as Tr{q)^i{q) = 7r(g — 1)A, we obtain that 


q<q^* 


-A7r(0) -F Y (A ^(9 - 1) - A7r(g)) = -A7r(g^. - 1), 
or — > A^7r(g^. - 1)^. (3.7) 

ai 


Now we find a lower bound (73 on ^2 by using the above upper bound on 7 r(( 7 ^* — 
^3 be the largest integer such that 


1) in (3.5). Let 


A V ®i \A*/ 2 2 

m=l ^ ^ 
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Then < ( 72 - We have that satisfies 


1 / T/ 93-9^*+^ / \ \ m 1 

^ \ n* I - 2 2 

1 — a 


A V ai \^J- 

m=l ^ 




-1 


X — fjL* 


<J^ 
V 


\ \ 93-9m*+1 


2 

1 — a 


^3 - 9m* + 1 ^ 

M* 

Since > 0, we note that q^ is at least 


1 + (A - fi*) 


2 

1 — a 


Therefore, 


log^ 

M* 


1 + (A - ^*) \l -y 


1 — a 


- 1 


1 , 


93 > 

log[l + evyf 


-log(l-^) 


1 — a 
2 

- 2 . 


Since ey = we have 


- 2 , 


^ log[l + a2V^[V]] o 
93 ^ ^ ^■ 


- log 1 - 


02 


Since ( 5 ( 7 ) >|>%>^>%we have that 


Q(7) > 2 


log [1 + ggyar [7^]] 


- 2 


As y 4 , 0, we note that log (1 — = 0 (\/f7). Hence, for the sequence 7 ^ with C{'^k) — 

□ 


c(A) = Vfc I 0, we have that Qijk) = ^ 

Remark 3.2.3. We note that as F J, 0, 7 r( 0 ) —)• 0. We also note that as F i 0, there exists a set 
of queue lengths, Qh, occurring with high probability (1 — V), such that //(g) —> A,Vg G Qh- If 
7 r( 0 ) —^ 0, then \Qh\ —)• 00 . Furthermore, for each q G Q/i, 7 r(g) = 0{-\/V). We also note that the 
stationary probability for each q £ Qh become equal as f/ 0. Then the average queue length is 

Remark 3.2.4. We note that the lower bounding technique in |7] can be used to obtain the Q, 
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lower bound by considering a uniformized version of Q{t). We outline this method in Appendix 3.B 


Using the stationary probability of the queue length has its advantages, since it gives us additional 
insights into the form of the optimal policy. 


Lemma 3.2.5. For INTERVAL-/iCHOICE-2-l, for any sequence of non-idling admissible policies 
jk with C{jk) - c(A) = f4 I 0 we have that Qijk) = 5^ - O (f/^“^log (^)), for 0 < <5 < 1. 


We note that INTERVAL-/iCHOICE-2-l is very similar to FINITE-^CHOICE-1 for which we recall 
that the asymptotic order was O (f^log (y))- However, for INTERVAL-/xCH0ICE-2-l, we are only 
able to show that the order is O (^^“‘^log (y)), where <5 can be made arbitrarily close to zero. 


Proof. We note that in this case there exists a policy 7, for which fk{q) = bi = bx,yq > 0, with 
(7(7) = c(A) and (5(7) = For F = 0, we note that the above policy is optimal. The solution 

to the tradeoff problem is similar to FINITE-/iCHOICE-l in which the average queue length increases 
but only to a finite limit as Vk i 0. Consider a particular policy 7 in the sequence 7^ with Vk = V. 
We have that 

00 

^ {c{iXk)-l{lXk))T^k^{k). 

k=0 ri'k>i>\ 

Let ^* = b\ + ey, where ey is a function of V to be chosen later. Then we have that 


V > 

> 

> 

V 

- > 

ruaev 


(cifik) 

ma Y “ ^A)7r^(fc) 
maCy E 

Y 


(3.8) 


where rria is the tangent of the angle made by the line passing through {b\,c{b\)) and (62,0(62)) 
with the line Z(/i). We proceed as in the proof of the asymptotic lower bound for problem FINITE- 
;uCHOICE-l. But unlike in FINITE-/iCHOICE-l, we note here that any service rate arbitrarily close 
to bx might be used by a policy 7. Intuitively, since Yliik>fi* should approach 0 as fC 0, we 

require that the choice of ey should be such that ^ | 0 as ^ | 0. 

Let q^* = inf {g : /i(g) > //*}. For q < q^*, ^{q) < fi* and therefore vr(g)A < 7r(g-F l)/i*. Hence, 
by induction we obtain that 


7r(g^* 


m) < 7r(g^*) 



for m E {1,.. .,qij,*}. 
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(3.9) 






From (3.8), we have 


7r(g) = 1- Y ^ 


V 


g<<?a* 


q><?u 


niaev 


Now from (3.9) we have 




7r{q^*) ( ^ I , hence we have that, 


<?<9m 


q=0 


1-^ X 


1 - 


y \ 




^ m=l 







i 



7r(g^*) V rna€vj ^l* - \ 

We note that as Tr(qu*) < vr((7) < ^ , we have that , ^ ,, > and therefore 


"laey f ^ _ AA^ < 

ruaev) ~ n* - X 


V 

log^ 



/f 


* \ 'Ja** 


- A niaev _ V I ^ 


- 1 


< V- 


12* y \ rUaev, 

By definition, for every q < q^*, fi{q) < jj*, and for every q > q^*,fL{q) < r^ax- Let us define 




log^y 


H* -Xmqey / _ V A ^ 

12* V rUaev) 


which is the smallest possible value for q ^* for any policy 7. Consider another policy 7' defined as 
follows : 

m = 0, 

fj'ig) = A, for 1 < 5 
fj-il) = '^max’) for q > q^yi. 

Then Q{'^') < Q{'l)- We now obtain a lower bound on Q{"i') as in FINITE-^CHOICE-1. 


Recall that for 7' we have (using the sequence of steps leading to (2.10)) 


<3(7') > (1-a) 

a 


L(l-a)2 


{1 - (1 - a){q^vi + <j q^yi-^Y + ^ 


> 


1 — a 
a 


+ (1 - a) <( qfayi- -+ 


1 — 6 (1 — 6)2 j (1 — a ) 


1-6 (1-6)2 


{(1 - a){q^vi + l)a'*'''7* + a.o'^^** 


1 — a 1 — a 


[1 + (1 - a)q^vi ], 
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where a = and b = :;r^- We note that for V iO, the term (l — d-ofeylV 

If y i 0, since we require that ^ i 0, q^*^i t oo and therefore the second term in the lower 
bound for < 5 ( 7 ') is aq^*. We note that at F = 0, since we require that the lower bound 
is tight, we only consider ey such that ey i 0 as F I 0. Then it can be shown that < 5 ( 7 ) > 
Qil') = ~ ^ {^v + ^ log ' fo'' sequence ey iO and ^ | 0 as F 4 , 0. By choosing 

ey = , where 0 < 5 < 1, we obtain that 

For the sequence we therefore obtain that Q{'^k) = 5 ^^ — O log (y))- □ 

Lemma 3.2.6. For INTERVAL-;uCHOICE-2-2, for any sequence of non-idling admissible policies 
7 fc with C'( 7 fc) - c(A) = Vfc 4 , 0, we have that Q{^k) = ^ (log (t^))- 


Proof. Consider a particular policy 7 in the sequence jk with I 4 = V- Let P* = o,\ — ^v- We have 
that 


^ (c(Afc) - l{pk))T^k^{k) -F ^ (c(^fc) - l{pk))Trt^{k), 

k-k<0.\ 

> Y (c(Afc) 

Mfc<“A-CV 

= l^a Y («A - Afc)7r^(A:), 

tJ'k<a\-ev 

> maeyPr{fi{Q) < n*} , 


where rua is the tangent of the angle made by the line passing through (a^ = a\,c{ai)) and 
(oj-i, c(aj_i)) with the line l{p). Define ( 7 ^* = inf {g : ii{q) > fi*}. As 7 is admissible, we have 
that Pr{iJ,{Q) < = Pr {Q < g^*}. Hence we have 


Pr{Q<q^*} < 
and 7r{q^* - 1) < 


V 

rriaey' 

V 

rriaev' 


We now choose ey = e, a positive constant. To find a lower bound on Q{'^), in the following, we 
intend to find the largest q such that Ylq=o^i9) — 2 - ^^it we note that Pr {Q < and 

for any q < q^*,Tr{q) < Therefore, 7 r(g^*) < 7 r(g^* - 1)^ < 777 ^- Let q^ be the largest 
integer such that 


9 


Y 


9=9^* 


1 V 

2 rriae' 
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then Qi < q. Proceeding as for problem INTERVAL-^CHOICE-1, we obtain a lower bound q 2 
qi by using an upper bound for 7r{q). We note that if g > q^* we have that 7r{q — 1)A > 7 r(g)^*. 
By induction, we obtain that for q 


Aq) < AQt^* - 1 ) 

q' — 

and for any q > q ^*, X] Aq) < - 1 ) 


\ <7-9^*+! 


m 
fi* 


Using the above upper bound on 'AAq=q * T^{q)< '^{qfi* — 1) < AtI’ obtain the following 

lower bound ^2 to 

If ^2 is the largest integer such that 


V 






E 

m=l 


A ]^ < 1 _ 
fl*J ~ 2 TTlae’ 


then YALo'^iq) — 5 W ^ Hence, q 2 is the largest integer such that 




- e, 

92 < 9m* - 1 + log^ 


< 1 + 

1 + 


A - OA + e\ enia f I V 


A 


V V 2 rriae 


A - oa + eA erua (I V 


V \ 2 m„e 


Since q^* > 0, ^2 is at least 


log^ 


1 + 


A - oa + eA erUa f I V 


A 


V V 2 m„e 


- 1 


Therefore, 


Q(y) > 1 > II > Is > 1 

\ — 2 — 2 — 2 — 2 


log^ 


1 + 


A - oa + eA ema /I f/ 


y V 2 m„e 


- 2 


Hence, for any sequence jk with (7(7^)—c(A) = f4 | 0, we obtain that QAk) = ^ (^log (tr)) ■ ^ 


Remark 3.2.7. We note that as f/ 4 , 0, 7r(0) i 0. We also note that as f/ 4 , 0, there exists a set 
of queue lengths, Qh, occurring with high probability (1 — V), such that Aq) ^ WxybxlAq £ Qh- 
If 7 r( 0 ) —> 0, then \Qh\ —t 00 . We note that for q 0 Qh, 7r{q) = OiV). Then from the birth 
death structure of Q{t) we obtain that the smallest queue length qmin G Qh has 7r{qmin) = C){V). 
Since for q G Qh, Aq) ^ [«a,&a] and the policies that we consider are monotone, the stationary 
probability distribution of the queue lengths in Qh can be observed to be geometrically increasing 
and then decreasing, which leads to the log (y) growth for the average queue length. 
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Remark 3.2.8. We note that the lower bounding technique in |;44j can be used to obtain the 
Q (log (^)) lower bound by considering a uniformized version of Q{t) as we discussed in Remark 
|3.2.4[ We again note that obtaining the stationary probability of the queue length, as in the above 
proof has its advantages, since it gives us additional insights into the form of the optimal policy. 

Lemma 3.2.9. For INTERVAL-^CHOICE-2-3, for any sequence of non-idling admissible policies 
jk with Ci-fk) - c(A) = 14 I 0, we have that Qi'jk) = ^ ■ 

Proof. Consider a policy 7 in the sequence 7^, with I4 = V- Let fi* = X — ey = ax — ey. Since 
cink) > l{Pk), we have 


^ (c(Afc) - l{pk))'^k^{k) > Y (c(Afc) - l{pk))T^^L{k) 

- Y Pk)'^iM(^) > maevPr{fi{Q) < fi*} , 

where nia is the tangent of the angle made by the line passing through (oj = ax,c{ai)) and 
(tti-i,c(aj_i)) with the line Let ey = a 2 V, where 02 is chosen so that a = 7^^ < 1- We 

note that 02 can be chosen such that a is arbitrarily close to zero. Define q^* = inf {q : fj,{q) > /i*}. 
Since 7 is admissible, we have that Pr {Q < = Pr {iJ,{Q) < /j*} < . We have 

Of 

Pr{Q<q^,-} < 

O' 

and 7r(g^* - 1) < -. 

To find a lower bound on Q{'y), in the following, we intend to find the largest q, such that 
— h Pr{Q < q/j,*} < ^ and for any q < g^*,7r(g) < Therefore, 

— 1)^ < If Qi is the largest integer such that 


^(q) <1- 


V 

niaev' 


then qi < q. Proceeding as for problem INTERVAL-/xCHOICE-l, we obtain a lower bound ^2 
qi by using an upper bound for vr((?). We note that if g > q^* we have that 7r(g — 1)A > 7r{q)fj,*. 
By induction, we obtain that for q 


( A 

7r(g) < 7r(g^* - 1) ( — 1 


q' 

and for any q > g^*, Y^ '^{q) ^ 

Q=‘lii* 



Using the above upper bound on 



7r(g) we obtain the following lower bound g2 to g^. If g2 
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is the largest integer such that 


92-9m*+1 

- 1) ^ 

m=l 



1 — a 

2 


then ^2 < 9 - 
We note that 


y > ^ rUaiX- fik)TTlj,{k), 

= X] "^a(A-/i(g))7r(g). 

q<i^* 


Again, since 'K{q)ii{q) = 7 r(g — 1)A, it follows that 


V > maX7r(qf,* - 1). 


Now if ^3 is the largest integer such that 

E 


<? 3 - 9 m *+1 / \ \m , 

K 1 ^ 1 — a 


m-aA ^ V/i" 
m=l 


then ^3 < 52 - We have that (73 satisfies 


^3 < 9m* - 1 + logW 


1 + 


ey maA I — a 

,,, , >»g[i + ¥V4^] 

+ -iog(i-T) 

Since q^* > 0, and ey = 02 we have that ^3 is at least 


log [1 + 771002 ^^] ^ 

-log(l-^) 


So that 


^ \ I y — 2 — 2 — 2 — 2 — 2 


log [1 + maa 2 ^] 


- 2 


Since log (l — = 0 (f/) as F | 0, we have that for any sequence 7 ^ with ( 7 ( 7 ?;) — c(A) = 




□ 


Remark 3.2.10. We note that as F 0, 7 r( 0 ) —> 0. We also note that as F I 0, there exists a 
set of queue lengths, Qh, occurring with high probability (1 — V), such that /i(g) —> AjVg G Qh- 
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If 7r(0) —)> 0, then \Qh\ — t oo. Furthermore, for each q G Qh,'^{Q) = rather than 0{W) 

as in Remark 3.2.3[ We also note that the stationary probability for each q £ Qh become equal as 
y I 0. Then the average queue length is Q (’^). We note that the difference from the behaviour 
in Remark 3.2.3 arises since c{fi) is piecewise linear rather than being strictly convex. 


3.2.2 Asymptotic behaviour of the tradeoff curve 

In this section, we obtain asymptotic upper bounds for the cases INTERVAL-/iCHOICE-l, INTERVAL- 
;uCHOICE-2-l, INTERVAL-/iCHOICE-2-2, and INTERVAL-;uCHOICE-2-3. With the asymptotic 
lower bounds which were derived in the previous section, these bounds provide an almost complete 
order characterization of the tradeoff curve Q*{cc). 


Lemma 3.2.11. For INTERVAL-/iCHOICE-l, there exists a sequence of admissible policies jk with 
a sequence Vk i 0, such that Qi'jk) = O log and C(7fc) - c(A) = I4. 


Proof. We evaluate the average queue length Q{'y) and average service cost C{'y) for a policy 7 
defined as follows : 


a(0) = 0, 

p{q) = A - 6(7, for g G {1,. .. ,gi}, 

fj.{q) = X + e'u, ior q£{qi + l,...,2qi}, 

p{q) = A + AT, for g G {2g'i + 1,.. .} . 


Let 6(7 = VU, e'jj = 571^, and K be some positive constant such that \ + K < r^ax- We also 


let qi = 


log 


sequence Uk i 0. 


(i + tfi) 


. The sequence of policies 7^ is obtained by choosing U from a 


For 7, we have 
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for q£{l,...,qi}, 
for q G {^1 + l,...,2gi}, 
for q G {2qi + 1,...} . 


















Now, since = 1 we have that 



7r(0) 


1 + — 


A 


X — eu 

7r(0) 


<?i 


1 + — 
eu 


-1 I + 
A 


A 


<?i 


A 


X — eu 


X — eu 

Ql 

-1 I + 


u 




A 


<?i 


A 


X — eu 


X + e'^ 

^__ 

‘'u ‘'n ^ K 


A A 


= 1 , 

= 1 . 


From the above expression, using the lower bound log/ A ~ 1 on qi, we have that 


7r(0) < 


1 + A ((^) (1 + a) - i) + 4 (1 + tft) 


i^'u + e)(A(l - U) - eu) + UXe'u{l + 
0{U) 



a’ 

K 


(3.10) 


We now evaluate (7(7). We have that 

C(7) = 


where and G{K) = c(A + K) — c(A) — Combining the c(A) terms, 

bounding 7r^(A — eu) + vr^(A + e^) + vr^(A + K) above by 1 and using eu = e^ = 0{y/U), we 
obtain that 

( 7 ( 7 ) ^ c(A) + 0(U) + {—eijT^fi{X — eu) + ej/7r^(A + e^) + W7r^(A + iC)) d" G{K)tt + W). 

In the following, we show that (5(7) < 00 and therefore 7 is admissible. Then, since 


7 r( 0).0 + 7 r^(A — eu)c(X — eu) + '^/i(X + e^[;)c(A + e(j) + vr^(A + K)c{X + K), 

T^fiiX — eu) ^c(A) — eu + C’(e^)^ + vr^(A + 6 ( 7 ) ^c(A) + e'u ^ + ©((ejy)^)^ 

+'?i'//(A + iC) ^c(A) + + G{K)'^ , 


7r^(A — eu){X — eu) + T^fiiX + e( 7 )(A + e^j) + vr^(A + K){X + K) — A, therefore 
7r//(A — eu){—eu) + 7r^(A + e'u){e'u) + 7r^(A + K){K) = 7r(0)A. 


Now from (3.10) we have that 7r(0) 
Therefore 


0{U). Furthermore, 7rfj,{X + K) = 7r(0)^ 


0{U). 


G{-f) < c{X) + 0{U) + ^^0{U)+GiK)0iU). 
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For V = C(j) - c(A), V = 0{U). 


with Qe = 2qi +1 and e = K, 

to obtain that 


In order to obtain an upper bound on (5(7), we use Proposition 


2.A.1 


Q{i) 

Qii) 


^ (2(7 i + l)(A'+ A) A + r, 

- K ^ 

= 0{qi) = 0 


Vu 


log 


2K 

1 

U 


Hence 7 is admissible. 


Corresponding to the sequence Uk, we have a sequence I4 = 0{Uk)- Therefore, we have a 
sequence of policies 7^ with ^(7^) = O (^log Then <5(7^) = O (^log and 

C{^)-c{X) = Vu. ' ' □ 


We note that the asymptotic upper bound above for the sequence 7^. does not match the asymptotic 
lower bound VL which was derived in Lemma 


3.2.2 


Lemma 3.2.12. For INTERVAL-/xCH0ICE-2-l, there exists a sequence of admissible policies 7^, 
with a sequence of I4 I 0, such that 5^ - <5(7^) = 0 (Vklog (t^)) and C{'yk) - c(A) = 14. 


Proof. Consider the policy 7 defined as follows : 


a(0) = 0, 

p{q) = bx, for q e {l,...,qbj, 
p{q) = r^ax, for q £ {qb^ + I,.. 


The sequence 7^ is obtained by choosing 


proof of Lemma 2.3.8 


k £ Z+. The rest of the proof is similar to the 

□ 


Remark 3.2.13. We note that the above asymptotic upper bound does not match the asymptotic 
lower bound in Lemma 3.2.5 Since ^{q) £ [0,rmax], one expects that perhaps another sequence 
of policies for which fi{q) = bx + ey, for g G {1 ,..., g^^} and where ey is a sequence decreasing to 
zero, achieves a better asymptotic upper bound. We have found out that this is not the case, and 
the asymptotic upper bound is the same as the one above. 


Lemma 3.2.14. For INTERVAL-/xCHOICE-2-2, there exists a sequence of admissible policies 7^, 
with a sequence I4 i 0, such that Qi'jk) = O ^log (tr)) ^nd C{'yk) — c(A) = I4. 
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Proof. Consider a policy 7 defined as follows : 


m = 0 , 

P{q) = OA, for g e . ,gi}, 
pig) = 6a, for g G {gi+ 1,...}, 


where gi = 


log 


1 + 


A-Qa 1 
A U 


, with U > 0. The sequence 7 ^ is obtained by choosing U 


from a sequence i 0. The rest of the proof is similar to the proof of Lemma 2.3.12 Since 


Vk = 0 (Uk), we obtain < 5 ( 7 ^) = O (log 


□ 


Using the asymptotic lower bound on Q{'yk) from Lemma 3.2.6 and the asymptotic upper bound 


above, and proceeding as in the proof of Proposition 2.3.9| we obtain the following result 


Proposition 3.2.15. For INTERVAL-/iCHOICE-2-2, we have that the optimal tradeoff curve Q*{cc,k) 
is 0 ^log (^ 7777 ( 7 ))), for a sequence Cc^ = C{'yk)< where 7 ^ is the sequence of policies in Lemma 
13.2.141 


Lemma 3.2.16. For INTERVAL-/iCHOICE-2-3, there exists a sequence of admissible policies 7 ^, 
with a sequence I 4 i 0, such that Q( 7 fc) = o(^j and C{'j) — c(A) = 14- 


Proof Consider a policy 7 defined as follows : 

a(0) = 0, 

P{q) = A, for gE gi}, 

p{q) = A + a:, for g E {gi + 1,...} . 


We note that A = ai, for some i > 1. Here iC is a constant such that A + iC < o^+i. We 
define gi = with U > 0. The sequence 7 ^ is obtained by choosing U from a sequence 
Uk i 0. The rest of the proof is similar to that of Lemma 2.3.14 Since I 4 = 0{Uk), we obtain 

□ 


Q{lk) ~ ^ ( Vfe ) ■ 


Using the asymptotic lower bound on Qi'jk) from Lemma 3.2.9 and the asymptotic upper bound 
above, and proceeding as in the proof of Proposition 2.3.9[ we obtain the following result 


Proposition 3.2.17. For INTERVAL-^CHOICE-2-3, we have that the optimal tradeoff curve Q*{cc,k 
^ I, for a sequence Cc,a: = C'( 7 fc). where jk is the sequence of policies in Lemma 


is 0 


^c,k ^(A) 


3.2.16 
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3.2.3 Asymptotic characterization of order-optimal admissible policies 


As in Chapter 2, it is possible to obtain an asymptotic characterization of any sequence of order- 
optimal admissible policies using the bounds P/{.} , {.} , 7r;(.), and tTu{-) and the inequalities 


(2.31) and (2.32). We only consider the cases where the minimum average queue length in¬ 
creases to infinity in the asymptotic regime 3?. The characterization of order-optimal policies for 
INTERVAL-//CHOICE-2-2 and INTERVAL-/iCHOICE-2-3 can be obtained using similar methods as 
FINITE-/xCHOICE-2 and FINITE-/iCHOICE-3. Therefore, in this section we discuss the asymptotic 
characterization of the optimal policy for INTERVAL-/iCHOICE-l only. We first obtain Pi {A} and 
Pu {^}, where A C [0,rmax] is a set of service rates. 


Let 0 < (5 < 1 and 02 > 0. Let A P [0, A 
that 


a2V^2^] U[A -F 021^^2^ 


From (3.1), we have 


Pr{f,{Q)eA}<Pu{A}^ 


0102 ’ 


if 5 > 0. Suppose <5 = 0, then for 02 > —, we have that 


Pr{fi{Q)eA}<Pu{A}^ 


1 

01 O 2 


(3.11) 


(3.12) 


We then note that Pi \AA = 1 —^ if <5 > 0 and Pi {AA = 1- — if 5 = 0 and 02 > —. 

^ L J aia2 ^ ^ J aia2 ^ ai 


The upper bound TTu{q) that we use in the following is obtained as in (3.4). We now proceed as 


in Chapter 2 to obtain a lower bound 7r;(0) on 7r(0). From Lemma 3.2.11 we have that for any 
sequence of non-idling order-optimal policies jk, Q{lk) = O ^;^^log Lise the above 

upper bound since we are not able to show that Qi'yk) = ^ ' Therefore, as in Section 

we have that 


2.3.3 


TT^O) = 



(3.13) 
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we then have that 


With k = - 

log 1 e 




TTl{0) = Q 


w 


1 


1 'l 

ru 


= n 


log (f) 


(3.14) 


Furthermore, we have the following lower bound 7ri{q') on 7 r(g'), where q' = max{g : fi{q) < A}. 

We note that for // < A, there exists a mi > 0 such that c(/i) — — Also for /i > A, 

there exists a m 2 > 0 such that c(//) — Z(/x) < m 2 (/U — A). Then we have that 

V = ¥.[c{^x{Q))-l{n{Q))\< ^ mi (A -/i(g)) 7 r(g) + ^ m 2 (//(g) - A) 7 r(g). 


V < mi E (A -//(g)) 7 r(g) + m 2 ^(//(g)-A) 7 r(g), 
<?<?' <?>g' 


= mi A 7 r( 0 ) + ^ (A 7 r(g) - A 7 r(g - 1)) 

< miA 7 r(g') + m 2 A 7 r(g'), or, 

./I A 1^ 


+ m 2 ^ ( 7 r(g - 1) - 7 r(g)) A, 
q>q' 


7r(g') > 7r;(g') = 


A (mi + m 2 ) 


(3.15) 


We note that q' could be zero, in which case the sum Yli<q<q' ~ ^'^{q ~ 1)) 'S defined to 

be zero. 


Lemma 3.2.18. For any sequence of non-idling order-optimal admissible policies 7 ^, with C{'yk) — 
c(A) = Ffc I 0, and Qa = {g : //(g) G for a ^ C ]^,rmax], we have that 


\Qa\ = < 

ifA={A}, 

where 0 < 5 < 1 , and 02 > -^ if (5 = 0 . 




^ (a) ’ 


0, A — 0214 

X-a2vP,X + a2vP 


(V) 


C A 


Proof. Consider a particular policy 7 in the above sequence with C{'yk) 


0, A — a2V^ 2 ) 


Then, from (3.11) we have that Pu{A} = 


— c(A) = V. Let A = 
Consider a g such that 
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^{q) G A. Then 7r{q) > TTi{q), where 


T^iig) = ^i(O) 


A-a2V(^)J 


and 7ri(0) is given in (3.14). As in (2.32) if q^ = max{q : ^{q) G A}, then the smallest i 
such that 


qi,u 




> 


q=0 


Z_ 

0.102 


is an upper bound on qi. We have that 






A-02F(V) 


-1 > 




l+<5 > 
2 > 


Oi (A-oaF^V)^ 


From (3.14), we have that for small enough V, there exists a 03 > 0 such that 


7ri(0) > 

So if q[^ is the smallest integer such that 


log (f) 


A 


A-ooF^V) 


- 1 > 


y( 




log(F) 

Ol (^A - 02!^^^)) + 0 

then q[ ^ > qi^u- Or we have that q[ ^ is the smallest integer such that 


log 


F*. 


' 1+5^ 




qIu > 


ai ( X—a 2 V 




log 


.X-a2v(^) . 

As F I 0, we have that q[ ,^ = O {ylog (^)). If Qa = {q ■ fJ^{q) G A}, then \ Qa\ = C> 


Now consider A = 


A — 02 ^( 2 )^ A + oaF^a) , where oa > ^. We have that P;(A.) = 

Let qi = max|g : fj,{q) < A — oaF^a) Then from (3.7) we have that 7r(g'i) < 
every q £ A, we have that 


7r(g) < TTuiq) = ' ^ 


q-gi 


VA-oaF(^) 


nteger qi^u 


^log(^)). 

: 1- 

aia2 

. Then for 


(3.16) 
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Let Qa = {q ■ fJ^iq) £ A}. We now obtain a lower bound on \Qa\- From (2.31), if®,a is the largest 
integer such that 


V 


A 


m,a 

g=0 ^ ^ \ A — 02 I 


< 1 - 


1 


0102 


VA-02^(5) 

then qi^a < \Qa\- This is equivalent to finding the largest integer qi^a such that 

A 


V 


Ql,a 


V VA-ooF^^) 


-1 < 1 - 


1 


a2V'^ 


A 


Ql,a 


A - 02 ^( 5 ) 


-1 < 1 - 


0102 7 A - O 2 F 2 
1 A oi\/fl2A^ 


01O27 A - 02F2 


(3.17) 


Then as in the proof of Lemma 


3.2.2 


we have that ®,a = ( ^ ). 


Consider Qx = {q '■ fi{q) = A}. We note that 7r{Qx) < 1. We also note that for every q G Qx, 
(q) < T^iiq) = T^i{q') defined in (3.15). As in ( 2.32D , if qu^x is the smallest integer such that 


TT 


qu,xT^i{q') < 1 , 

then q^^x is an upper bound on IQaI- Flence we obtain that |Qa| = C> (y). 


□ 


Remark 3.2.19. From the above proof, the intuition behind choosing the buffer partition to scale 
as Q. can be observed. 


3.2.4 Tradeoff problems which are similar to INTERVAL-/iCHOICE 

In this section, we consider tradeoff problems which are similar to INTERVAL-/iCHOICE, for 
which an asymptotic characterization can be obtained using the techniques presented above for 
INTERVAL-//CHOICE. We note that for INTERVAL-/xCHOICE, we restricted to admissible policies 
7 for which \{q) = A,Vg G Z+, and A was such that u{\) > Uc- The counterpart INTERVAL- 
ACHOICE of INTERVAL-/ 1 CHOICE is one in which we restrict to admissible policies 7 for which 
^x{q) = /X, Vg G {1, 2, ...} and \{q) G [0, ra 

We note that C(7) for such a policy 7 is (1 — 7r(0))c(/x), which depends on the policy, unlike 
INTERVAL-/iCHOICE where the choice of A fixed 77(7) to be u{\). For INTERVAL-ACHOICE, 
we restrict to admissible 7 such that ^{q) = /x,Vg > 0, where /x is such that c(/x) < Cc, so that 
^( 7 ) < Cc. The tradeoff problem INTERVAL-ACHOICE is 

minimize ..ygr^ Q{l) 

and U{'y)>Uc- (3.18) 
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The optimal value of the above problem is denoted as Q*{uc)- We also note from Lemma 2.2.2 that 
the maximum value of U (7) over all admissible 7 is u{^). We obtain an asymptotic characterization 
of Q*{uc) in the asymptotic regime 3?, where Uc t 

We assume that /x is such that Va^max > If ra,max = /U. then we note that only the non- 
admissible policy 7, with \{q) = ra^maxj'^q, can achieve Intuitively, if Uc t u{^), then since 

EA((5) = E^(Q) = (1 — 7r(0))/x t A*, wc have that 7r(0) 0. Then for any policy which is feasible 


as Uc t u{X), 7r(0) | 0. Thus, intuitively for problem (3.18) we do not have a case where Q*{uc) 
increases only up to a finite value as Uc t (unlike INTERVAL-/xCHOICE-2-l). 

As for INTERVAL-/iCHOICE, we consider the following cases for INTERVAL-ACHOICE: 


INTERVAL-ACHOICE-1: u{X) is strictly concave for A G [Q,ra^max]- 


INTERVAL-ACHOICE-2 : u{X) is piecewise linear and concave. That is, (a) there exists a minimal 
partition of [0,ra,max] into intervals {[a*, 6 *], A G {1,...,P}} with ai = 0, 6p = Vmax, and 
bi = Oj+i and (b) there are linear functions fi such that Va* G [ai,bi], fi{X) = u{X). This is 
further subdivided into two cases: 

1 . = Gi < ^ < bf^ = bi, for some A > 1. 

2. AX = = Gi, for some A > 1. 


Remark 3.2.20. We note that for the discrete time queueing model, we only consider the case where 
the utility function is linear, since for such models we are interested in the average throughput as 
the performance measure. We note that INTERVAL-ACHOICE-2-1 encompasses the case of linear 
utility functions. For linear utility functions, we can motivate the choice of system parameters for 
INTERVAL-ACHOICE-2-1 as done for the case of INTERVAL-axCHOICE. INTERVAL-ACHOICE-2- 
1 with linear utility function is a simplified model for a discrete time queueing model, where the 
service batch size is fixed (but if the queue length is less than this fixed batch size, then the service 
batch size is equal to the queue length) and there is randomized admission control. The fixed 
service batch size is modelled by the fixed fi, while the admission control is modelled by the choice 


of the arrival rate X{q) as a function of q. In light of the discussion in Section 3.1 we assume that 
X{q) takes values in a finite interval. 


We now present an asymptotic characterization of Q*{uc) in the regime Uc t We note that for 
INTERVAL-ACHOICE-1 as well as for INTERVAL-ACHOICE-2 it is possible to show (see Lemma 


for every Uc < u{^), Ve > 0, there exists some feasible 7 G Tq such that <5(7) < Q*{uc) -F e. Such 
an admissible policy is called e-optimal in the following. 

We first present asymptotic lower bounds on Q*{uc) in the regime Uc t Asymptotic lower 

bounds for INTERVAL-ACHOICE are obtained along similar lines as for INTERVAL-axCHOICE. For 


3.2.28) that there exists a sequence of admissible policies 7^ such that t/(7fc) t ^(a)- We note that 
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all cases, we first obtain upper bounds on the stationary probability of certain arrival rates (rather 
than service rates), which go to zero as Uc t ^(/^)- Then as before, these upper bounds on the 
stationary probability of certain arrival rates lead to constraints on the stationary probability of all 
queue lengths. Since the stationary probability of the queue length determines the average queue 
length, the constraints determine the behaviour of average queue length as Uc t 

For the asymptotic analysis of INTERVAL-ACHOICE, we define a line 1{X) whose definition is similar 
to that oi l{fi) for INTERVAL-^xCHOICE. For INTERVAL-ACHOICE-1, f(A) is defined as the tangent 
to n(A) at A = /U. For INTERVAL-ACHOICE-2-1, 1{X) is defined as the line through (a^, u{a^)) and 
while for INTERVAL-ACFIOICE-2-2, /(A) is any line through (a^,u(a^)) with slope m, 
such that |;v=/i < m < We note that Z(A) > u(A) and El{X{Q)) = Z(EA(Q)). 

We also note that the function Z(A) — u{X) is a convex function. 

We now present a result, which formalizes the intuition that 7r(0) 0 as ttc t 


Lemma 3.2.21. For INTERVAL-ACHOICE, for any sequence of admissible policies such that — 
U{Xk) = Tfc 4, 0, we have that 7r(0) = C>(f4). Therefore, as Uc t 7r(0) | 0, for any sequence 


of feasible policies for (3.18). 


Proof. Consider a particular policy 7 in the sequence with I4 = We note that U{'y) < u(EA((5)). 
Since 7 is admissible, we have that U{'y) < u(E^{Q)). We then have that u~^{U{'y)) < E/i((5) = 
(1 — 7r(0))/i, since exists if it(A) is concave and increasing in A. Therefore, we have that 


We note that u~^{x) > x G M+, where ) is the inverse function of Z(A). Then we have 

^ ^ - V) ^ ^ _ l-^ juju)) - mV ^ 


A 


A 


A 


since = Z(^) and where m is the slope of l~^. Therefore, for the sequence 7^, '7r(0) = 0(14). 
For INTERVAL-ACHOICE, as Uc t u{p), for any sequence 7 ^ of feasible policies, tt(//) — 0 ( 7 ^) | 0 
and hence 7r(0) i 0. □ 


For any policy, the set of arrival rates {X{q) : q G Z+j is countable and is denoted as (Aq, Ai, ...), 
with Afc < Afc_|_i. For an admissible policy, let 'n'x{k) denote the stationary probability of using an 
arrival rate Xk, i.e., 7rx{k) = Pr {X{Q) = A^} = J2{q-.\{q)=x^} 

We make the following assumption, which is similar to (C2): 

U2: For INTERVAL-ACHOICE-1, the second derivative of tt(A) at A = /i is non-zero. 
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Lemma 3.2.22. For INTERVAL-ACHOICE-1, for any sequence of non-idling admissible policies 
such that u{p) — U^'jk) = f4 0, we have that Qi'jk) = Therefore, Q*{uc) = 

ni , _ 

y'up)-'L 


Proof. The proof follows that of Lemma 3.2.2 but with some minor differences. We again consider 


a particular policy in the sequence with Vk = V. Let X* = p + ey, where ey > 0 is a function of 


V to be chosen later. As in the proof of Lemma 3.2.2 we have that 


q=0 


(a “ Kq)) ^ lA=/i + G{X{q) - p) 


where G{x) is a strictly convex function of x. As X{q) — ^ is bounded, we again have that there 
exists a positive ai such that 


V > ^7r(q) 

q=0 

Since — P’ 


(a - Mq)) + ai(A(g) - pf 


V > Y^7r(q)ai(X(q) - pf 

q=0 


A . 


Let qx* = inf {q : X{q) < A*}. We note that unlike qfj,* in Lemma 
as in the proof of Lemma 


3.2.2 


qx* could be 0. We proceed 


3.2.2 


by choosing ey = a 2 VV. Then Pr {Q < qx*} < if qx* > 0. 


As before, we choose 02 such that Pr {Q < qx*} < §, where a can be made arbitrarily close to 
zero. If qx* = 0, then Pr {Q < qx*} = 0 < 

we find the largest q such that Pr {Q <q}<^. We note that if 


3.2.2 


As in the proof of Lemma 
q> qx*< then TT{q — 1)A* > 7r(g)//. Then by induction we obtain that for any q > qx*, 

q q-q\* 

vr(g) < TT{qx*) Y 


q=q\* 


m=0 


y\ m 

A 


(3.19) 


We note that this is similar to (3.4), except that we express the above upper bound in terms of 
7r{qx*) rather than 7r(g^* — 1) in (3.4), since qx* could be zero. 


If qx* = 0, then from Lemma 3.2.21 we have that n^qx*) = vr(0) = 0{V). If qx* > 0, we obtain 


an upper bound on Triqx*), as in the proof of Lemma 3.2.2 We have that 


V 

— > 
ai 


Y ^( 9 )(^( 9 ) - a )' 


q<q\* 


> I Y “ A)7r(g) 

P<qx*-l 
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Since for q > 0, since '7r{q)X{q) = fnT{q + 1), we proceed as in the proof of Lemma 3.2.2 to obtain 
that 

— > ifJ.7r{qx*) - , 

ai 

= fi^7r{qx*f + /.i^vr(O)^ - 2fi‘^7r{qx*)7r{0). 


Since 7r(0) > 0 and 7r(0) = 0(V) from Lemma 3.2.21 we have that 


ai 


+ 2^i^7r{qx*)7r{0) > ij.^7r{qx*y, 

-+2^l^O{V) > fi\{qx*f, 
oi 


or 


7r((ZA*) = 0{VV). 


We note that for both qx* = 0 or qx* > 0, we have that TT{qx*) = 0{VV). 


We now proceed as in the proof of Lemma 3.2.2 by using ( |3.19D , to find the largest integer q such 
that 




Yj 


m=0 


y\ m 


^ 1 a 
-2~ 2' 


The rest of the proof is similar to that of Lemma 


3.2.2 


and we obtain that Qi'fk) = ^ (t^)' 


Then given a sequence of Uc^k t u{n), we have that there exists a sequence of feasible 7 ^ such 

that QHk) < Q*{uck) + e, for some e > 0. Therefore, Q*(uck) = ^ , I ], since 

y ^(m) '^c,k J 

Uc,k<U{-fk). □ 


Lemma 3.2.23. For INTERVAL-ACHOICE-2-1, if bi = and i < P, then for any sequence of non¬ 
idling admissible policies 7^ such that u{iJ,) — U{'yk) = f4 4- 0, we have that Qi'jk) = ^ (^log (\t)) ■ 
Therefore Q*iuc) = n (log (77^)). 


Proof. The proof follows that of Lemma 3.2.6 We define A* = bn + e, where e > 0. Let 


qx* = inf {q : X{q) < A*}. We note that qx* could be 0, unlike q^* in Lemma 3.2.6 


If qx* = 0, then we have that T^iqx*) = vr(0) = 0{V). If qx* > 0, then we have that 


^ = E mk)-u{Xk))nx{k), 

A(j>A* 

> rriaePr {X{Q) > X*} , 


where rria is the tangent of the angle made by the line passing through {bi = and 

{bi+i,u{bi+i)) with l{fj,). Or we have that Pr {Q < qx*} < ^ and 7r{qx* - 1) < Since 
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^{qx* — l)A(gA* — 1) = 7r(g'A*)A^ we have that •K{qx*) < 'K{q\* — , We note that therefore 

= C){V) for both qx* = 0 and qx* > 0. 


Now proceeding as in the proof of Lemma 3.2.6 we have that for any q > qx* (we express the bound 
in terms of TT{qx*)) 


Q /\*\™ 

vr(g) < 7r(gA*) Y (v) • 

q=q\* m=0 A /^ / 


We note that independently of whether qx* is 0 or not, if we find the largest q such that 


^ , 1 V 

^ - 2 -^’ 


q=q\* 


then <5(7) > I. We now proceed as in the proof of Lemma 


3.2.6 


using the upper bound in (3.20) 


and 7r(gA*) = 0{V) to obtain that Q^'jk) = ^ (^log Now given a sequence of Uc^k t 

we have that there exists a sequence of feasible 7^ such that Q{^k) < Q* {uc,k) + (-, for some e > 0. 


Therefore, Q*{uc,k) = ^ (log (7(7)3777)). since Uc,k < U{jk)- 


□ 


Remark 3.2.24. We note that the above lemma is used to obtain an asymptotic lower bound on 
Q*{uc)- As far as this asymptotic lower bound is concerned, the above lemma can be used even 
when if i = P, where i is such that bi = b^. We consider INTERVAL-ACHOICE-2-1 for a larger Xx 
defined as follows. We extend Ta to ^La = ^La U {bp, bp + 5], for some <5 > 0. We also extend the 
definition of u{.) to Xx, by choosing a piecewise linear function on {bp, bp + 5] which preserves the 
strictly increasing concave property of «(.). We denote Q*{uc) when X{q) G Xx, by Ql{uc)- Then 
we note that Q%{uc) < Q*{uc). The asymptotic lower bound for Q%{uc) follows from the above 
lemma, which then also holds for Q*{uc)- 


Remark 3.2.25. We note that the above asymptotic lower bound holds even in the case where 
= 0. For INTERVAL-/iCHOICE-2, we note that the case with ca = 0 corresponds to the case 
INTERVAL-//CHOICE-2-1, for which Q*{cc) only increased to a finite value. 

Remark 3.2.26. We note that in many cases, for queueing models with a single queue, the utility 
constraint is on the average throughput. Then we have that u{X) is a line segment, with = 0 
and b^ = ra,max- We note that the asymptotic O (log (y)) lower bound holds for Q*{uc), from 
the discussion in Remark 13.2.241 


Lemma 3.2.27. For INTERVAL-ACHOICE-2-2, for any sequence of non-idling admissible policies 
7fc such that «(//) — f7(7fc) = 14 0, we have that Q{'yk) = ^(17) ■ 


Proof. The proof follows that of Lemma 3.2.9 We choose X* = ^ + ey = an + ey- Let qx* = 


inf {g : X{q) < A*}. We note that qx* could be 0, unlike in Lemma 3.2.9 
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If q\* is 0, then we note that TT{qx*) = 7r(0) 
that 


0{V) from Lemma 3.2.21 


If qa* > 0, then we have 


^ iliXk)-u{Xk))7Tx{k), 

Afc>A* 

> rUaePr {X{Q) > X*} , 


where nia is the tangent of the angle made by the line passing through (a* = a^,u[amu)) and 
{ai+i,u{ai+i)) with /(^). Or we have that Pr {Q < qx*} < and TT{qx* - 1) < We 

also note that since Tr{qx* — l)X{qx* — 1) = TT{qx*)lJ. w® ^^^ve that TT{qx*) < vr((?A* — . 


Therefore 'n'(qx*) < 


V 


' a,max 


ma€v M 


. We choose ey = a 2 V, so that 


' a,max 


ma^a2 


< f, where a « 1. 


Then, as in the proof of Lemma 3.2.9 if q is the largest integer such that 


y 1 

E / \ ^ 

Mt) 


q=q\* 


then (5(7) > independently of whether qx* = 0 or not. 
We note that for any q > qx* we have that 


Q Q—Qx* / \ * \ ™ 

Triq) < 7r{qx*) Y (~p) ' 

i—n. ^ -yv,—n \ ^ / 


We also note that if qx* > 0, then we have that 

y > Y IT^aiXk - fJ-)TTx{k), 

Afe>A* 

g«ix* 

= maHTT{qx*) - fJ.TT{0). 


Then since 7r(0) = 0{V) we have that 7r{qx*) = 0{V). Thus independently of whether qx* = 0 or 
not, we have that T^iqx*) = 0{V). 


Now proceeding as in the proof of Lemma 3.2.9 using the above upper bound on ir{qx*) in (3.21) 
we have that Q{'yk) = L2 Now given a sequence of Uc^k t we have that there exists a 

sequence of feasible 7^ such that Qijk) < Q*{uc,k) + e. for some e > 0. Therefore, Q*{uc,k) = 
since We,fc <C/(7fc)- □ 


We note that as for INTERVAL-//CHOICE, using policies with similar structure as in Lemmas 3.2.11 


3.2.14 and 3.2.16 it is possible to obtain an asymptotic upper bound for INTERVAL-ACHOICE-1 
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and tight asymptotic upper bounds for INTERVAL-ACHOICE-2-1 and INTERVAL-ACHOICE-2- 
2. Here, we obtain a single asymptotic upper bound for INTERVAL-ACHOICE-1, INTERVAL- 
ACHOICE-2-1, and INTERVAL-ACHOICE-2-2, which shows that = sup^gp^ 

Lemma 3.2.28. There exists a sequence of admissible policies 7 ^ such that Q^'jk) = O 
u{n) - Ui'jk) = Vk- 


Proof. The proof follows that of Lemma 3.2.16 (which follows from that of Lemma 2.3.14). Consider 
a policy 7 defined as follows: 


/X for g e {0 ,... ,gi} , 

Kq) = { 

[fi-K for q G {qi + 1,...} , 

where qi = ^ for positive U. The sequence of policies 7^ is obtained by choosing U from a 
sequence Uk i 0. 

We note that 


[7(7) = Pr{Q<qi}u{n)+Pr{Q>qi}u{n-K), 

= u{n) - Pr{Q > qi}{u{n)-uifi-K)). 

Oru{n)-U{'y) = Pr {Q > qi} {u{iJ.) - u{n - K)). 

We also note that Q{'y) = O (qi) = O (^). Therefore we have that Pr {Q < qi} n+Pr {Q > gi} 
K) = (1 — 7r(0))/x. Or 


(1 - Pr{Q> qi}) fi + Pr{Q > qi} {jj. - K) 
IJL + Pr{Q> qi} {-K) 
Pr{Q > gi} 


(1 - 7r(0))7X, 

— 7t7r(0), or 
A7r(0) 

K ■ 


Following the proof of Lemma 2.3.14 we have that 7r(0) = O (U). Hence, we have that Pr {Q > qi} 
0{U). Therefore, for the sequence of policies 7^, we have that u{^) — U{'jk) = 0{Uk). If 
Vk = u{p) - Ui'yk), then we have that Qi'jk) = O □ 


Remark 3.2.29. Since the techniques used in the analysis of INTERVAL-^xCHOICE and INTERVAL- 
ACHOICE are similar, we expect that asymptotic bounds on any sequence of order-optimal policies 
can be obtained for INTERVAL-ACHOICEas for INTERVAL-/xCHOICE. We note that the role of 
fj.{q) and X{q) are interchanged. For example, using a similar sequence of steps as in the proof 
of Lemma |3.2.18[ it is possible to show that for a sequence of non-idling order-optimal admissi¬ 
ble policies with — U{'yk) = f4 | 0, \Qa\ = INTERVAL-ACHOICE-1, where 

Qa = : X{q) G [fJ. — 02^2, ^ + 02145] I and 02 > 0. 
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3.3 Analysis of INTERVAL- A/iCHOICE 


We recall that for INTERVAL-A/xCHOICE we restrict to policies 7 such that X{q) G [ra,min,ra,max] 
and G [0,rmax]< Vq G Z_|_. The tradeoff problem for INTERVAL-A/iCHOICE is: 

minimize Q{l) 

such that ^(7) < Cc, 

and 17(7) > Uc, (3.22) 


whose optimal value is Q*{cc,Uc}. Although it is possible to consider various forms of the function 
as in the case of FINITE-/iCHOICE and INTERVAL-//CHOICE, here we obtain a complete 
analysis for the case c(/i) being a strictly convex function of ^ G [0,rmax] (with assumption C2) 
and u{X) being either a strictly concave (with assumption U2) or a piecewise linear function of 
A G [ra,min,ra,max]- The reason for this assumption is the motivating discrete time problem, 
described below. We then comment on the asymptotic bounds for other forms of c(^) in the 
following discussion. 


Remark 3.3.1. INTERVAL-A/iCHOICE corresponds to the tradeoff problem for the following dis¬ 
crete time queueing model. Work arrives in a batch of random size, in every slot, into an infinite 
buffer queue. The queue length is the amount of unfinished work and evolves on M_|_. The amount 
of work which is admitted into the queue can be controlled, as a function, possibly randomized, 
of the current queue length. This feature is modelled by the control of the arrival rate, X{q), in 
INTERVAL-A/iCHOICE. The amount of work done by the server in each slot, or the service batch 
size, can also be chosen as a function, possibly randomized, of the current queue length and is as¬ 
sumed to be a non-negative real number. This feature of the discrete time queue is modelled by the 
control of the service rate, fi{q), in INTERVAL-A//CHOICE. We note that the drift in the discrete 


time queueing model is real valued. In light of the discussion in Section 3.1 we assume that X{q) 
and fj,{q) take any non-negative real values, but in finite intervals. In each slot, assume that there 
is a utility accrued with admitting customers and a service cost incurred in serving them. These are 
modelled by the utility rate and service cost rate functions u{.) and c(.) in INTERVAL-A/iCHOICE. 
We choose c(/u) to be strictly convex as for INTERVAL-/iCHOICE-l. Another motivating factor for 
considering c(-) to be strictly convex, is the need to explain the logarithmic 0 (log (^)) behaviour 
of the average queue length when admission control is allowed, noticed by Neely in |44J, compared 
to the Q, behaviour of the average queue length, with strictly convex c(-), when admission 

control is not allowed. We note that in j44j, since there was only a constraint on the average 
throughput, the function u{.) need only be linear, while the following result is presented for the case 
of strictly convex or piecewise linear u{.). 

Remark 3.3.2. In this analysis, we assume that Uc < u{ra,max)- If Uc > u{ra^max), then there 
does not exist any feasible policies for (3.22) We note that if Uc = u{ra,max), then policies which 
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satisfy this utility constraint need to have X{q) = ra^max,'^Q, in which case the problem is the same 
as that considered in INTERVAL-/iCHOICE-l Va^max < fmax)- We also note that the restriction 
of analysis to admissible policies implicitly requires that u~^{uc) < Vmax- 

Lemma 3.3.3. For INTERVAL-A/xCHOICE, with Va^min > 0, the service cost C{'y) for any admis¬ 
sible policy 7 is bounded below by c{u ^(Uc)). 

Proof. To find a lower bound on the service cost for any admissible policy 7, we consider the follow¬ 
ing equivalent formulation of TRADEOFF ( |2.4D and use a series of relaxations on the constraints. 
The minimum average service cost for a given average queue length constraint Qc and a utility 
constraint Uc is given by 


min E^cf/ifQ)), 

7era 

such that E7ru(A((5)) > Uc, and E^Q < qc- 

We note that for every qc < 00, we have that E7rA(Q) = E7r/x((5). So the optimal value of the 
optimization problem above is bounded below by the optimal value of 

min ET,c{fj.{Q)) , 

such that E7rw(A((5)) > Uc, and E7rA((5) = E.n-;u((5). 

Since u{X) is concave in A, we have that for every 7 such that E^m(A(Q)) > Uc, rx(E7rA(Q)) > Uc- 
Therefore the optimal value of the problem above is bounded below by the optimal value of 

min E^c(/i((3)), 

TT 

such that E7rA(Q) > u~^{uc), and E7rA((5) = E7r/i(<3), 

where we are considering all possible distributions vr for Q. Now since c{^) is convex in we obtain 
that the optimal value of the above problem is > c{u~^{uc)). Therefore C{'y) > c{u~^{uc)). □ 

In the following, as in the case of FINITE-/xCHOICE and INTERVAL-|uCHOICE, we consider 
INTERVAL-A/iCHOICE in the asymptotic regime 5R where the service cost constraint Cc approaches 
c{u~^{uc)), where Uc is kept fixed. 


3.3.1 Asymptotic lower bound 


In this section we find an asymptotic lower bound on Q{^k) foi" any sequence of non-idling admissible 
policies 7fc for which C/(7fc) > Uc and C{'yk) i c{u~^{uc)). Subsequently, in Lemma 


3.3.7 


we 


show that there exists a sequence of non-idling admissible policies 7^ for which C{'yk) approaches 
c{u~^{uc)) arbitrarily closely. 
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We note that, as in the case INTERVAL-/xCHOICE, even though the service rate iJ,{q) can take any 
value in [0,rmax] and the arrival rate \{q) can take any value in [ra,min, ra,max], the sets of service 
rates and arrival rates used by 7 are both countable as the queue length takes only integer values. 
We first present the asymptotic lower bound for the case when Va^min > 0, for which the proof is 
very similar to the case INTERVAL-/iCHOICE-l, and then consider the case ra,min = 0. 

Lemma 3.3.4. For INTERVAL-A//CHOICE, for any sequence of non-idling admissible policies 7^ 
such that C'(7fc) — c{u~^{uc)) = 14 | 0 and U{'^k) > Uc, we have that 


Proof. Consider a particular policy 7 in the sequence 7^ with I4 = V- Since 7 is admissible, we 
have that = E,rA(Q). From the concavity of u(A), we have that E.n-A(Q) > u~^{uc). Let 

jj,* = u~^{uc)—ev, where ey is a function ofV to be chosen later. Define q^* = inf {q : /j,{q) > fj,*}. 
We note that V(? < q^*, ^{q) < /i*. As ^{q) is non-decreasing, we have that 

Pr{Q < q^,*} = Pr{ix{Q) < p*] . 


Let the countable set of service rates be denoted by {/io = 0,/^i,...}, where < //j+i and 
Pi £ [0,rmax]- Let l{p) be the tangent line at (M“^(nc),c(tt“^(uc))) to the curve c{p). Then 
^ = E^o [c(a(9)) - Kp{q))] From Proposition 


3.A.1 


we have a positive oi such that 




.-1 


V > 01 ^^ [p{q) - u ^(uc)] vr(g) > ai ^ [p{q) - u ^(uc)] 7r{q). 

q=0 q=0 


Flence 


Pr{Q < q/,* - 1} < - 

V 

and 7r(g^* - 1) < -o'- 

«i4 

Now, since Q{t) is a birth death process Vq, we have that 7r{q)X{q) = 7r(g -F l)p{q -F 1). For any 
q>qti*, 

( I 1\ ^ 7r(?)^a,max 

'^{Q + 1 = —f —fit - -J-’ 

p{q+l) p* 

<q) < - 1) j • (3.23) 

Let q be the largest integer such that Ylq=o'^(^) — 5- Fnd a lower bound on q as in the proof 
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of Lemma 
For V sma 


2.3.6 


We note that Pr {Q > > 1-Let ey = e, where 0 < e < u 


, let Qi be the largest integer such that 


V 


^ 2"a^' 


Then < q. We find a lower bound on by using the upper bound on 7r{q) from (3.23). Let q 2 
be the largest integer such that 


- 1 ) 


92-'?m*+1 

E 

9=1 




1 

<- 


V 


2 aie^' 


Then ^2 E Qi- After substituting for /i*, we have that any ^2 satisfying the above inequality is such 
that 


92 - 9m* + 1 ^ log 




1 + 


ra,i 


— It 


-1 


(itc) + ey 


1 


' a^max 


1 


Hence we obtain that ^2 is at least 


log 


‘^a.jTnaa: 
u~'^{uc) — eY 


1 + 


' a^max 


- u ^{Uc) + ey 1 


7r(9M* - 1) V2 


V 


V 

aiey 


7r(g^. - 1) V2 


aiey 


- 2 . 


We note that 


7r(Qu*-l) 


> and is the dominant term in the regime where V I 0. Since 


9 > 9i > 92 and Q{'j) > we have that for any sequence of 7^ with C{'yk) — u ^(ttc) = 14 | 0, 


Q{lk) = ^ (log 


□ 


Remark 3.3.5. We note that in this proof, no use was made of the assumption that the sequence of 
policies satisfies the constraint E,r^i(A((5)) > Uc- The difficulty in problem INTERVAL-A/xCHOICE, 
is to actually construct such a sequence of policies. 

In the above proof, we note that there exists a set Qh of queue lengths occurring with high 


probability, such that ^{q) 


u 


-1 


(lie), for every G as lA | 0. But for every q G Qh, it is 


possible to ensure through arrival rate control that the service rate //(g) is not equal to the arrival 


rate A(g), while for Lemma 3.2.2 //(g) = A for all queue lengths with high probability. Hence, for 
admissible policies, for g G Qh, the stationary probability distribution geometrically grows and then 
decays which leads to the log (y) behaviour. 


' a,min — 0 OC A(g) G ■ 


Remark 3.3.6. The case when 
We note that if ra^min = 0, then the birth death process may not be irreducible on Z+. Therefore, 
in this case, admissible policies are assumed to induce a single positive recurrent class including 

> 0. \f Va^min = 0, then for an admissible policy 


zero. In Lemma 3.3.4 we had assumed that r„ 


7, there could exist g such that A(g) = 0. Let q' = inf {g : A(g) = 0}. Note that q' is in general 
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dependent on the parameters Cc and Uc- If q' is infinity, then the same approach as in Lemma 


3.3.4 holds. If q' is finite, then as 7 is admissible, \/q > q', \{q) = 0. Hence all states q > q' are 


transient and in steady state we need only consider the CTMC evolving on {0,... ,q'}. Note that 
Y11=o'^{q) — f therefore q in Lemma 3.3.4 is smaller than q'. Therefore, the same approach 
as in Lemma 3.3.4 holds even if q' is finite. 


3.3.2 Asymptotic behaviour of the tradeoff curve 


In this section, we construct a sequence of admissible policies jk which achieves the minimum 
average service cost c{u~^{uc)) arbitrarily closely with Qijk) scaling at the optimal rate as in 


Lemma |3.3.4[ However, we are able to obtain an asymptotic upper bound only for the case where 
u{X) is strictly concave or linear (and not piecewise linear). 


Lemma 3.3.7. For INTERVAL-A//CHOICE, with u{X) strictly concave or linear, there exists a 
sequence of admissible policies 7 ^ with a corresponding sequence f 4 | 0 such that 

<3(7.) = 

C{jk) - c{u~^{uc)) = Vk, 

U{-fk) > Uc- 


The construction of the sequence of admissible policies 7 ^ is motivated by the following intuition. 


that we have obtained from the lower bound in Lemma |3.3.4[ The sequence of policies should 
be such that as Vk i 0 , the service rate used, at a queue length occurring with high probability, 
should be close to u~^{uc)- But the arrival rate X{q) should not exactly equal u~^{uc), for all 
queue lengths q which occur with high probability. Then it should be possible to have a stationary 
distribution which is geometrically growing and then decaying, leading to the required log ^^ 7 :^ 
scaling of Q{jk)- 


Proof. Consider a policy 7 of the following form : 


and 


a(o) =0, 

P{q) = Pi= u~^{uc) - eu, for g G {1,..., gi} , 
p{q) = P2= u~^{uc) + eu, for g G {gi + 1 ,.. .} ; 
X{q) = Ai, for g G {0, - 1}, 

X{q) =u~^{uc), for q £ {qi,...,qi+ K} , 

X{q) = X2, for q £ {qi + K + 1 , .. .} . 
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Let €[/ = [/, Ai > u ^(uc) > A2 ,Ai >/ii,A2 <//2, and (l + , We will 

specify K, Ai, and A2 later. Let obtain C{'j). 

Cin) = 7r(0).0 + 7r/,(^i)c(^i)+ 7r^(^2)c(/i2), 

= 7r^(w) + 0(4)) + 

vr^(^2) ^c(rx"^(wc)) + + 0(6^)) , 

< c(«"^(nc)) + 0(t/^) + (-ef/7r;,(^i) + euTT^j,{^i2)) ^^^^ 

< c(u“^(nc)) + 0(0), 

where = ^^|^=„-i(„^). Let F = 0(7) - c(ii“^(uc)), then we have that V = 0(0). 

For 7, the average utility is 

( gi — l \ gi+J^ 00 

^ 7r(g) + Wc J]] 7r(q) + u(A2) ^ 7r(q). 

q=0 J q=qi q=q^+K+l 


Let Xi = u ^{uc) + e and X 2 = u ^{uc) — e, where e is a small positive constant. Then for strictly 
concave and linear u{.) we have that 

Ql —1 00 

0 ( 7 ) > Uc+ e^7r(g)-e ^ 7r(g) 0)(ii(«“^(uc))) + 0(it(ii“^(uc))), 

g=0 q=qi+K+l 

where D{u{u~^{uc))) and 0{u{u~^{uc))) are defined as follows. If u{.) is a strictly concave func¬ 
tion, then it is differentiable at u~^{uc) and D{u{u~^{uc))) = ^7 ^|a=m-i(«c) ^nd 0(ii(«“^(itc))) = 
O(e^), with the above inequality being an equality. If u{.) is linear then D{u{u~^{uc))) = 
'^^^^^\x=u-^(uc) ^nd 0{u{u~^{uc))) = 0, with the above inequality being an equality. 


In the following we show that Yl^qi+K+i ^(9) — X)g=o^ which case we have that 0(7) > 

Uc for sufficiently small e (which is fixed and independent of V). We have that 



for g G {!,... ,gi} , 
for g G {gi -F 1,..., gi -F W} , 
for g G {gi -F or + 1,...} . 
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Therefore, 




Ai f ^ ^{Uc) \^ f u ^{Uc) 

/^2 / \ fJ ‘2 — ^2 




< 


Ai 


Ai — /ii 



-l)+l (3.24) 


then Er= 


q=gi+K+l 


7 r(q) < Yll=o^ '^°te that (3.24) can be simplified to the question 



U ^{Uc)\^ ( u ^{UcY 

/^2 — A2 , 


/^2 


U ^(Uc) 





u ^{UcY 
u-^{uc) + euJ \ e + eu , 



u ^{Uc) 


- 1 + 


et/ 


u ^(Uc) 


-K^ 


We use the lower bound on qi, obtained by removing the ceiling, to arrive at the following question 


u ^{Uc) - €u e + €u 1A / u _ eu _ \ 

e + eu “V u~^{uc)-euU J \ e + eu J \y u-^{uc) V u-^(uc) J J 

For sufficiently small U, with eu = U, we have that < 1 — 2 u-+u ) • 'f^stead 

of the above question we can ask the stronger question 

u-^{uc) - eu ^ A e + eu 1 \ f eu (^ 

e + eu “V u-^{uc) - euV J \ e +eu J \u-^{uc) \2 ))' 
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We choose K > 2 [1 + ——). Then we can ask the even stronger questions 


-1/ N ^ /^-i , e-heu 1\ f (u \uc)f 


-euV 


u {uc) - eu < 


e 

-If 


e + eu {u ^{uc)) 
e u-^{uc)-eu 


which indeed hold. Hence for sufficiently small V and e, U{'y) > Uc- 
We now obtain Qij) using Proposition 


2.A.1 


with = qi + 1. Hence, 


, {qi + l){€u + e + 

’^a.max) '^max 

Q[v = -^-h 


£[/ + e 


2 (e[/ + e) ’ 


7 =:/ \ ^ {,Ql T l)(e T fa,max) fmax 

Qh) < -^ 


As = O (log (p)), we obtain that Q{'y) = O (log (^)). We note that the policy 7 is admis¬ 
sible. The sequence of policies is obtained by choosing Uk = p We note that then we have a 
corresponding sequence I4 = 0{Uk)- Thus, Qijk) = O ^log (^tr))' there exists 

a sequence of admissible policies 7^ with a corresponding sequence 14 | 0 such that 


Qb,) = o(iog(T)), 

Cilk) - c{u~^{uc)) = 14 , 

u (jk) > Uc- 


□ 


Remark 3.3.8. We note that the above proof also applies if u(A) is piecewise linear and {u~^{uc),Uc) 
lies on a linear segment of the piecewise linear function ti(A). However, the proof does not apply if 
u{X) is piecewise linear and u~^{uc) is such that the slope of ti(A) changes at (u~^(uc),Uc)- 


Using the asymptotic lower bound from Lemma 3.3.4 the asymptotic upper bound above, and 


proceeding as in the proof of Proposition 2.3.9 we arrive at the following result. 


Proposition 3.3.9. For INTERVAL-A/iCHOICE, for strictly concave or linear u(A), we have that 
the optimal tradeoff curve Q*(cc,k,Uc) = O (^log ■ for the sequence Cc,k = C(jk), 


for the sequence of policies 7^ in Lemma 3.3.7 


Remark 3.3.10. For INTERVAL-A/iCHOICE, an admissible policy 7 can be specified by the sets 
Qfi,x = {q. '■ ^(9) = A)-^(9) = for all possible /x and A. However, we are only able to obtain 
bounds on sets of the form Qa = {q ■ lJ-{q) G 4 C [0,rmax]} for an asymptotic characterization 
of a sequence of order-optimal admissible policies. These bounds can be derived using similar 
techniques as in Section 13.2.31 For example, if Vamin > 0, then it can be shown that if 4 C 
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[0,ti ^{uc)—a 2 V 2 ]|J[m ^{uc)+a 2 V 2 ], for 0 < <5 < 1 and 02 > 0, then (^log 

for a sequence of non-idling order-optimal admissible policies with (7(7^) — c{u~^{uc)) = 14 | 0. 


3.3.3 Tradeoff problems which are similar to INTERVAL-A/xCHOICE 


In this section, we first consider other asymptotic regimes for (2.4), which are similar to INTERVAL 


A/iCHOICE. We note that for INTERVAL-A/xCHOICE, the utility constraint Uc was kept fixed while 
Cc,k i c{u~^{uc)). A similar problem (SPl) is one in which Cc is fixed and Uc,k t u{c~^{cc)). Another 
problem scenario (SP2) is one in which both Cc^k and ttc,fc vary such that (a) Cc,fe — iO 

or (b) w(c“^(cc,fc)) - Uc,k I 0. 

We note that SP2(b) encompasses SPl since the sequence Uc,k can be chosen such that Uc,k = 
Uc,yk G Z+. We now show that the asymptotic regime for SP2(b) is equivalent to that for SP2(a), 
i.e., Cc,fc — c i 0. We note that for any (uc^k) and (cc,fc), for which the problem (3.22) is 


feasible, and also such that u{c ^(cc,fc)) — Uc,k i 0, we have that Ve > 0, 3K^ such that, V/c > K^, 


u{c ^(cc^k)) — e < Uc,k < ^(cc,fc)) (since Uc^k < uic ^(cc,fc)) if the problem (3.22) is feasible). 


Then we have that u ^ [u{c ^(cc,fc)) — e) < u ^{uc,k) < c ^(cc,fc). For every Cc,a:, we define 


^i,fc(A) (as in Section 3.2.4) to be (i) the tangent to u{X) at (c ^{cc^k),u{c ^(cc,fc))), if m(A) 


strictly convex, (ii) the line passing through {a^,u{af^)) and (6^,u(6^)), if m(A) is piecewise linear 
and c“^(cc,A;) lies on a linear segment, and (iii) any line through {a^,u{a^)) with slope m, such 


that 

d\ 


|a=/j < m < |A=/i, if u{X) is piecewise linear and c ^{cc^k) is a corner point of 


u{X). We note that li^kic ^{cc,k)) = u{c ^{cc^k)) in all three cases. Then 


■ ^ (n(c ^(cc,fc)) - e) > j (^i(c ^(cc,fc)) - e) = j {u{c Hcc,fc))) - 

where mi^k is the slope of Since i{u{c l(Ce, k))) = c ^(cc,fc), we have that 

c~^icc,k) - rni^ke < u~^{uc,k) < c“^(cc,fc), 
c (c“^(cc,fc) - m^fce) < c{u~^{uc,k)) < Cc,k- 


Let l2,k{k) be the tangent to c(^) at (c ^{cc^k)^ Cc,k)■ Then we have that 

h,k {c~^{cc,k) - mi^ke) < c{u~^{uc,k)) < Cc,k, 

Cc,k - m2,kmi^ke < c{u~^{uc,k)) < Cc,k, 


where m 2 ^k is the slope of l 2 ^k- We note that 3Mi, M 2 G M+ such that mi^k < Afi and m 2 ,fc < M 2 
for every k, since both u{X) and c(^) are defined on bounded domains. Since the above inequality 
holds for every e > 0 and k > K^, we have that that Cc,fc — c{u~^{uc^k)) i 0. 

We now have the following result, under the stronger assumption that u{.) is m-strongly concave. 
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with m > 0. The proof is similar to that of Lemma 3.3.4 


Lemma 3.3.11. For INTERVAL-A/iCHOICE, for any sequence of non-idling admissible policies 7 ^ 
and a sequence Mc,A: > 0 such that C{'^k) — c{u~^{uc^k)) = f4 | 0 and t/( 7 fc) > Uc,fc. we have that 

1 


Q{-ik) = O ( log ( — 


Proof. The proof follows that of Lemma 3.3.4 closely. Hence, we only state the differences here. 
We define fj,* = u~^{uc,k) — and q^* = inf {q : fi{q) > //*}. We note that unlike in the proof of 
Lemma 3.3.4| we define a different tangent line 4(/u) for every Uc,k- Let 4 (a) Le the tangent line 


to c(a) at {u ^{uc^k),c{u ^{uc^k)))- From Proposition 3.A.1 we have a positive ai^k such that 


Vk > ai,fc ^ [fi{q) - u~^{uc,k)] 7 r(g). 

q=0 


A . 


We note that unlike the proof of Lemma 3.3.4 here ai^k depends on the sequence Uc,k- Let 


a = inffc {ai^fc}. Since u{.) is m-strongly convex, we have that a > m > 0. Then we have that 


Pr {Q < q^* - 1} < 


V 


ae 


2 ’ 


V 

V 


and 7 r(g^* - 1 ) < 


ae 


V 


Then, we proceed as in the proof of Lemma 


3.3.4 


to obtain that ( 5 ( 7 fc) = n ^log □ 


We note that an asymptotic upper bound can be obtained by evaluating Q( 7 fc), C'( 7 fc), and U{'yk) 


for a sequence of policies 7 ^ as in Lemma 3.3.7 but with Uc now being the sequence Uc^k- Then 
we have the following result 

Proposition 3.3.12. For INTERVAL-AaCHOICE, for strongly concave or linear u{X), we have that 
the optimal tradeoff curve Q* icc,k,Uc,k) = 0 (^log ( c, fc)) )) ’ sequence Cc,fc = 

C'( 7 fc) and Uc,k = U{'yk), for the sequence of policies jk as above. 


Remark 3.3.13. We note that for INTERVAL-AaCHOICE we have considered the case where 
c(a) is strictly convex and u{X) is either strictly concave or linear (also piecewise linear for the 


asymptotic lower bound in Lemma 3.3.4). Although we have not presented the analysis for other 


forms of c(a), such as when c(a) is piecewise linear, here we outline how the methods presented in 
Chapter 2 as well as this chapter can be used in obtaining asymptotic lower bounds in these cases, 
in the asymptotic regime where Cc,fc i c{u~^{uc)). Suppose c(a) is piecewise linear and Uc is fixed. 
We note that as in INTERVAL-|uCHOICE-2, we can define service rates fii and a^ with respect 
to u~^{uc) rather than A. Then the asymptotic behaviour of Q*(cc,Uc) depends upon whether 
(i) A« < < Pu and pi = 0 or (ii) otherwise. For case (i), we note that Q*{cc,Uc) only 


111 













increases to a finite value, since we can fix X{q) = u~^{uc) and apply the analysis of INTERVAL- 
//CHOICE-2-1. However, we do not have an asymptotic lower bound in this case. For case (ii), we 

except that jx* = m — e, where e > 0, to obtain that 

Q*{cc,Uc) is Q (log (^)). 


can proceed as in the proof of Lemma 


3.3.4 


3.4 Conclusions 


In this chapter, we have considered the asymptotic characterization of the tradeoff problem for the 
state dependent M/M/1 model, with model features chosen such that insights can be obtained for 
the tradeoff problem for discrete time queueing models also. From the analysis, we see that for 
INTERVAL-/iCHOICE and INTERVAL-A/iCHOICE the constraint on the average service cost, in 
the regime SR, leads to constraints on the stationary probability distribution of the service rate fx{Q) 
and therefore the stationary probability distribution of the queue length which in turn determines 
the asymptotic growth of the minimum average queue length as a function of the average service 
cost. The exact nature of these constraints and the behaviour of the stationary distribution of the 
queue length depends on the nature of c(/i) at = A and the extent of freedom in the choice of 
X{q) and /i(g) at a queue length q. 

We observe that if c(/i) is strictly convex at A, then in the asymptotic regime SR as 1/ = Cc —c(A) | 0, 
the stationary probability of any queue length q such that //(g) / A goes to zero. More precisely, 
as y I 0, /i(g) for q e Qh (the set of queue lengths with high probability) has to approach A, the 
stationary probabilities for such queue lengths are equal and each is 0{VV). This leads to the 
behaviour for strictly convex c{fi). We expect that this is the same phenomenon which gives rise 
to the Berry-Gallager asymptotic lower bound [7], but for admissible policies. In Chapters 4 and 
5, we see that this particular behaviour of the stationary probability does carry over to the discrete 
time model. Suppose, however that it is possible to control the arrival rates A(g), as in the case 
of INTERVAL-A/iCHOICE. Then even though //(g) for queue lengths q £ Qh have to approach a 
single value u~^{uc), the drift for such g need not be zero, since A(g) can be chosen to be different 
from u~^{uc) for such g. In fact it is possible to chose A(g) such that the drift is initially positive 
and then negative for the set of queue lengths occurring with high probability so that the stationary 
probability of the queue length has a geometrically increasing and decaying behaviour which leads 
to the log (y) asymptotic growth for the minimum average queue length. We expect that this is 
the reason for the 0(log(y)) behaviour observed by Neely tA4j, but for admissible policies. In 
Chapter 5, we shall obtain an asymptotic lower bound for the discrete time model considered in 
[44j using the above idea, for admissible policies. 

When c(//) is piecewise linear in //, we note that either (A,c(A)) can be a corner point as in 


112 



INTERVAL-|uCHOICE-2-3 or (A, c(A)) can lie on a linear portion of c(;u) as in INTERVAL-^uCHOICE- 
2-1 or INTERVAL-/iCHOICE-2-2. For INTERVAL-^CHOICE-2-3, as in the case of INTERVAL- 
/iCHOICE-1, we observe that as f/ 0, fi{q) for queue lengths in Qh has to approach A, we again 
observe that the drift (proportional to A — fi{q)) for such queue lengths approaches zero, and the 
stationary probabilities for such queue lengths are equal and each is 0{V). This leads to the y 
behaviour for INTERVAL-/iCHOICE-2-3. 

For INTERVAL-/iCHOICE-2-l and INTERVAL-;uCHOICE-2-2, we note that as f/ | 0 service rates 
H such that fJ-i < fJ- < fJ-u could be used, so that the drift for the set of queue lengths occurring with 
high probability is not zero. For INTERVAL-/iCHOICE-2-2, it is possible to choose fj,{q) so that 
the drift is initially increasing and then decreasing for the set of queue lengths occurring with high 
probability. Then the stationary probability has a geometrically increasing and decaying behaviour 
which leads to the log (^) behaviour. We expect that this is the reason behind the O (log (y)) 
growth observed for the case of piecewise linear cost functions in li43j. For INTERVAL-^uCHOICE- 
2-1 we note that as F i 0 the service rate 0 could be used, which for non-idling admissible policies 
implies that the queue length 0 has positive stationary probability, even for V = 0. This intuitively 
implies that the average queue length does not increase to infinity in this case. 


Asymptotic bounds on any sequence of non-idling order-optimal admissible policies have been pre¬ 


sented in Section |3.2.3[ We have also discussed other variants of the tradeoff problem, such as 
INTERVAL-ACHOICE, where only the arrival rate can be controlled. Even though INTERVAL- 
ACHOICE is very similar to INTERVAL-/iCHOICE we find that there are no cases for INTERVAL- 
ACHOICE where the minimum average queue length increases to a finite value unlike INTERVAL- 
^uCHOICE. We observe that the asymptotic behaviour of the minimum average queue length as 
the utility constraint is made arbitrarily close to the maximum value of the utility, can be obtained 
using ideas which are similar to that of INTERVAL-//CHOICE, which shows that the method of 
obtaining the asymptotic behaviour of the average queue length through its stationary distribution 
for monotone policies is sufficiently general and provides a unified method which explains other 
scenarios also. 


We also note that through the analysis of the state dependent M/M/1 model, with proper choice 
of model features, we get insights as to how to construct asymptotic lower bounds for the discrete 
time models. In Chapter 4, we find bounds on the stationary probability distribution of the queue 
length for the discrete time queueing model in the regime 3?, which have the asymptotic behaviour 
as suggested by the above analysis. Thus these bounds lead to the right asymptotic behaviour of 
the average queue length in the regime 3? for the discrete time queueing models. 
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Appendices 


3.A A lower bound for strictly convex functions 

Proposition 3.A.I. Let g{x) : [0,i?] —)■ M+ be a finite strictly convex function, such that: (i) 
5 ( 0 ) = 0, (ii) g'{Q) > 0, and (iii) g"(0) > 0. Then there exists a positive constant a such that 

g{x) > ax^, Vx G [0,-R]. 

Proof. We note that at 0, g{0) = a.O. If 3a > 0 such that for all x G (0,i?], g'(x) > 2ax, then 
g(x) > ax^. So we have to prove that for all x G (0,i2], g'(x) > 2ax. In essence, we have to 
prove that it is possible to find a positive a such that Vx G > a. We note that if 

then it is possible to find such an a. 

Now we prove that inf 3 ;g(o,it] ^ Since both ^'(x) and x are non-negative, > 0. 
Suppose we assume that infa,g(o,ij] = 0. Then we have that Ve > 0, 3x such that < e. 

Consider a sequence i 0, then there exists a sequence Xn such that ^ < e^- Note that as 

Xn < R, we have that 1) lim^^oo g'{xn) = 0 and 2) lim„_^oo = 0. The sequence Xn may not 
be convergent. So we consider the subsequence ym such that limm^-oo Um = limsup^^o^ x„ = y. 
Note that g' {ym) = 0 and limm^-oo = 0 as is a subsequence of Xn- Now there 

are two cases : 1) y > 0 and 2) y = 0. Let y > 0, then by continuity of g'{x) we have that g'{y) = 0. 
But g'{y) > 0 for every y > 0 and we have a contradiction on the assumption that inf 3 .g(o,R] = 
0. Consider the second case when y = 0, then we have that = 0. But note that 

lirrijj-^o = g”{^) > 0- Thus we again have a contradiction. Hence inf 3 ,g(g^^] > q vVe 

choose a = I in46(o,fl] □ 


3.B Application of the Berry-Gallager lower bounding technique [[7] 
in Remark for INTERVAL /iCHOICE-l 


For INTERVAL-yCHOICE, since we assume that fi{q) < Vmax, we can obtain a discrete time 
process Qd['m],m G Z+, by uniformizing the CTMC Q{t) at rate = A -F Vmax as in Appendix 


2.A We now outline how the lower bounding technique in [7] can be applied to Qd['m] to obtain 
asymptotic lower bound for Q*{cc) as Cc i c(A). We first show that a slightly 


the 1 , ^ ^ 

y/cc-c{\) ^ 

modified form of [71 Lemma 4.1] holds for the uniformized process Qd[m] under an admissible 
policy 7 (we note that we are using admissibility as defined in Chapter 2 and not as in [Tj). As 
in [71 Appendix A], we have that Pr {Q < [ 2 ( 5 ( 7 )]} > Let qp = argmaXgg|Q |' 2 Q{ 7 )]} ^(9)' 


Then we have that ^{qp) > 


2 ( 20 ( 7 )] ■ 


Now we define Qd[ixi] = ioaax{Qd['m\,qp). Then as in [71 
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Appendix A] we can show that 


7r(gp)E Qd[rn + l] - Qd[rn]\Qd[rn] = Qp + ^ TT{q) {X - ^{q)) < 0. 

q=qp + l 


We note that E 


Qd[m + 1] - QdimWQdim] = qp 


= Therefore we obtain that 


A 


T^{<lp)—+ ^ 7r(g)(A-/i(g)) < 0, 


or, 


q=qp + l 


A 


Then to obtain the asymptotic lower bound on Q*{cc) we use the above upper bound in step (49) 
in the proof of [71 Proposition 4.2]. 
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CHAPTER 4 


On the tradeoff of average queue length and average service cost 

for discrete time single server queues 


4.1 Introduction 


In this chapter we consider the tradeoff between average queue length and average service cost for 
the discrete time single server queueing models introduced in Chapter 1. Such discrete time models 
are in some cases more appropriate for modelling resource allocation problems in communication 
networks than the continuous time models considered in the previous chapters. A context in which 
such a model and the following analysis may be appropriate is that of wireless networks with fading, 
where one of the issues, which has been studied by many researchers (see (7||, [2S|, |ISj, and j77j), 
is the optimal tradeoff of the average power and average delay, when the service batch size is 
dynamically chosen as a function of the fade state and the queue length. The characterization of 
the tradeoff between average error rate and average delay, for a point-to-point noisy link, when 
the service batch size is dynamically chosen as a function of the queue length, is another resource 
allocation problem that motivates the model studied in this chapter. The tradeoff of average power 
and average delay is dealt with in more detail in Chapter 5, whereas the tradeoff of average error 
rate with average delay is considered in Chapter 6. In this chapter, we consider two simplified 
discrete time models, with no admission control (i.e. A[m] = and a single environment 

state, to develop the basic techniques for the characterization of such tradeoffs. The glossary of 
notation that we use in this chapter is given in Table [TT| We now summarize the methodology 
that is used for obtaining the asymptotic lower bounds on the tradeoffs. 
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Symbol 

Description 

m 

slot index 

A[m] 

random number of arrivals in slot m (after admission control) 

A 

■^max 

maximum number of arrivals in any slot 

A, 

mean and variance of ^[1] 

5[m] 

batch service size in slot m 

Smax 

maximum batch service size 

ea 

probability of A[m] exceeding Smax 

Q[m] 

queue length at the start of (m + 1)*^ slot 

a\m] 

history of queue evolution 

7 

policy - (5[1],5[2], • • •) 

r 

set of all policies 

Ts 

set of all stationary policies 

Qii) 

average queue length for a policy 7 

c{s) 

service cost for l-model; a function of integer valued batch service size s 

Cr{s) 

service cost for R-model; a function of real valued batch service size s 

C{l) 

average service cost for a policy 7 

Cc 

average service cost constraint 

/3c. 

non-negative Lagrange multiplier 

c/ 3 .. {q, s) 

single stage cost; defined as g -|- /3c.c(s) 

* 

9/3 

optimal average cost for unconstrained MDP 

Jp{q) 

relative value function for unconstrained MDP 

* 

optimal stationary policy for unconstrained MDP 

s}iq) 

action at state q for the optimal policy 7^ 


recurrence class for 7^ 

Ta 

set of all admissible policies 

Hq) 

average service rate at a queue length q 

TT 

stationary distribution for a policy which is clear from the context 

TTry 

stationary distribution for policy 7 

Q*{Cc) 

minimum average queue length over the set under constraint Cc 

Si 

largest service batch size < A at which slope of c(.) changes 

Su 

smallest service batch size > A at which slope of c(.) changes 


Table 4.1: Notation used in this chapter. 


118 




4.1.1 Methodology 


In this section, we summarize a scheme that is used in Chapters 4, 5, and 6 to obtain asymptotic 
lower bounds on the minimum average queue length in the regime 5? as a function of the average 
service cost constraint (Chapters 4 and 6 ) or average power constraint (Chapter 5). As in Chapters 
2 and 3, we obtain a lower bound on the stationary mean queue length using an upper bound on the 
stationary probability distribution of the queue length under the assumption that the queue length 
process is a DTMC and ergodic. The average service cost (or average power) is then related to the 
stationary probability distribution. 


Let Q ~ vr, where tt is the stationary probability distribution of the queue length. From Markov 
inequality, we have that > qPr {Q > q}. Suppose Pr^iQ < q} is any upper bound on the 
stationary probability distribution Pr {Q < q}. If q is the largest q such that Pru{Q < q} < a, 
then we have that Et^Q > 9 (1 — «)■ Foi' convenience, we choose a = ^. 


The upper bounds Pru{Q < q} were obtained for the state dependent M/M/1 models in Chapters 
2 and 3 using the detailed balance equations and lower bounds on the service rate /i(g). In Chapters 
4, 5, and 6 we obtain geometric bounds on Pru{Q < q}, the first (Lemma |4.3.5 ) of which has 
been obtained by assuming certain properties for the transition probability distribution while the 


second (Lemma 4.3.7) has been obtained by extending the results available in Bertsimas et al. 
and 190. 


We now illustrate this scheme for the case of integer-valued queue evolution through an example. 
We will obtain that for the queueing process, the upper bound Pr„{.} has a geometric form, i.e., 
Pru {Q < q} = 7 r( 0 )/?'^. For our tradeoff problem, it will turn out that a non-negative function D{q) 
can be obtained, such that E^I1((5) is the difference between the average service cost and c(A) 
(which can be defined similarly as for INTERVAL-//CHOICE). Then Kt^D{Q) < 1/ for a 17 > 0, 
then we have that 7 r( 0 )Zl( 0 ) < V, where 11(0) > 0. Therefore, we have a further upper bound 
on the stationary probability distribution, Pru{Q < q} = Then, the largest qi such that 

Pfu {Q < q} < ^ satisfies < 5 - Or we have that = 

regime IR, as F | 0, we obtain that Ip is 0 (log ('7)) and therefore so is the average queue length 
as a function of V. Variations of this basic method are used throughout Chapters 4, 5, and 6 to 
obtain the asymptotic lower bounds. 


log 


(PM) 

P\2V I 


In the asymptotic 


4.1.2 System model - Integer valued queue evolution 

We assume time to be slotted, with slots indexed by m G Z+. In each slot m, a random number 
of customers A[m] G Z+ arrive into the system. The arrival process {A[m],m > 1) is assumed 
to be HD with A.[l] < A^ax, batch arrival rate EA[1] = A < 00 , var(A[l]) = < 00 . The 

customers arrive into an infinite buffer queue. In slot m, a batch of customers of size Slm] G Z_|_ 
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is served. The batch of S[m] customers is removed from the queue at the end of the slot just 
before the new batch of customers which arrive in the slot, A[m], is admitted. We assume 

that ^[m] < Smax, where Smax is the maximum batch size that can be served. We also assume 
that 


A1 : Pr{A[l] > Smax} > ea>0. 


We note that the above assumption is similar to the assumptions made in the definition of admissible 
policies in [7j and |43]. Furthermore, we note that the above assumption is reasonable, since the 
maximum number of arrivals usually exceeds the maximum capacity Smax of service, but A < Smax- 

The number of customers in the queue at the start of the (m + 1)*^ slot is denoted by Q[m]. We 
assume that QjO] = qo £ Z+ customers. The queue evolution for m > 0 is given by: 

Q[m + 1] = Q[m] — S[m + 1] + A[m + 1], (4.1) 

where S[m + 1] < min(5max, Q["i]). We define a policy 7 to be the sequence of batch sizes 
(S[l], S'p],...) under which the queue evolves according to ( |4.lD . The set of all policies is denoted 
by r. A policy 7 is stationary if S[m + 1] = S{Q[m]),\/m, where S{q) is a randomized function of 
q. The set of all stationary policies is denoted as T^. 

We assume that a service cost of c(s) is incurred when serving a batch of size s. For example, this 
cost could be the expected number of symbols in error for the transmission of a batch of message 
symbols. The function c(s) : {0,1,..., Smax} —t M+ is assumed to satisfy the following properties: 


Cl : c(0) = 0, 

C2 : c(s) is non-decreasing and convex Q for S £ {0, . . . , Smax 2}. 


We now define the performance measures that we are interested in: a) 
queue length for a policy 7 G r is 

Q[0] = qo , 

and b) the worst case average service cost for a policy 7 G T is 

Q[0] = qo 


C{'y,qo) = limsup —E 

M^oc J-yj- 


M 


c{S[m]) 


m=l 


Q{l,qo) = limsup ^E 

M^oo 


E<3[ 

m=0 


m\ 


The worst case average 

(4.2) 

(4.3) 


^If Smax = 1, then there is no tradeoff. If Smax > 2, then c(s -I- 2) — c{s + 1) > c{s + 1) — c(s),Vs £ 

, . ■ . 5 Smax 2 }. 
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4.1.3 System model - Real valued queue evolution 


We state only the differences from the model discussed in Section 4.1.2 We assume that the 
arrival random variable A[m] G [0,Amax] C M+, with mean A < oo and variance < oo. We also 
assume that the service batch size S'[m] G [0, Smax] C M+. Similar to assumption Al, we make the 
assumption: 


RAl : Pr{A[l] - Smax > (^a} > Ca, 


where both 6a and €a are positive real numbers. Let the initial queue length be go £ 1^+- We 
have that the queue length Q[m] G M+ and the evolution of Q[m] is given by (4.1) with 5[m] < 
min(S'max, Q[m — 1]). We assume that there is a service cost cr{s) associated with the service of a 
batch of size s. The function cji{s) : [0,5m,ax] — 1 1^+ is assumed to satisfy the following properties: 


RCl : cr{0) = 0, 

RC2 : cr{s) is strictly convex and increasing in s, for s G [0, 5max]- 


In the following, the model with integer valued queue evolution is referred to as the l-model, while 
the model with real valued queue evolution is referred to as the R-model. We note that R-model 
with the strictly convex cji{s) cost function is usually used as an approximation to the l-model, 
which has c(s) as the cost function. 

We note that our l-model is a simplified version of the model studied by Goyal et al. [25], wherein 
there is an additional state variable which is used to model fading, the arrival process is Markov, 
and Smax = oo. Our R-model is a simplified version of the model studied by Berry and Gallager 
[7], wherein there is an additional fade state variable, and Smax = oo. 


4.1.4 Overview 


The tradeoff problem ( |1.7[ ) for l-model and R-model is formulated as a constrained Markov decision 
problem in Section [A2l We consider the l-model first. For l-model, for certain values of the cost 


constraint, we consider an equivalent unconstrained Markov decision problem (as in (1.8)) in Section 
[4.3.1 We also identify several properties that are possessed by any stationary deterministic optimal 
policy for this problem. We then define the set of admissible policies, which are policies possessing 
the above properties (the definition of admissible policies is similar to that in Chapter 2). 


From Section 4.3. 2| onwards, we consider the tradeoff problem for the set of admissible policies. In 


Section 4.3.3 we characterize the infimum of the average service cost over all possible admissible 
policies, which is equal to the minimum average service cost required for mean rate stability. We 
identify three cases, which are similar to the three subcases for INTERVAL-/xCHOICE-2, for which 
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the asymptotic behaviour of the minimum average queue length is characterized. The asymptotic 
behaviour of the minimum average queue length is obtained as for INTERVAL-/iCH0ICE-2, by first 
obtaining upper bounds on the stationary probability of the queue length. One of these bounds is 
a state dependent extension of the geometric bounds on the stationary probability of discrete time 
Markov chains presented in |8!] and [9J. In Section 4.3.4 we show that depending on the value of 
the arrival rate, the minimum average queue length either (i) increases only to a finite value, or 
(ii) increases as log (^), or (iii) increases as y, when the average service cost is V more than the 
infimum of the average service costs for admissible policies. Asymptotic bounds on order-optimal 


policies are presented in Section 4.3.5 We obtain an asymptotic lower bound on the minimum 


average queue length for ergodic arrival processes in Section |4.3.7[ when the average service cost 
is V more than the infimum of the average service costs for admissible policies. 


For R-model, we present an asymptotic analysis in Section 4.1.3 We obtain that the minimum 
average queue length is (^) when the average service cost is V more than the infimum of the 
average service costs for admissible policies, for strictly convex cr(s). Then we consider the case 
where cr{s) is piecewise linear and show that the asymptotic behaviour of the minimum average 
queue length is similar to that of the l-model. 


4.2 Problem formulation for l-model and R-model 

The tradeoff problem is to obtain Q*(cc, go), which is the optimal value of the optimization problem 

minimize Q(7, go) such that (7(7, go) < Cc, (4-4) 

7er 

where Cc > 0 is the average service cost constraint. The tradeoff curve Q*{cc,qo) is non-increasing 
and convex in Cc (see [6J) for any go. We note that if A > Smax then any feasible policy for ( |4.4[ ) is 
optimal, as the average queue length for any such feasible policy is infinity. Hence, in the following 
we assume that A < Smax- 


4.2.1 A constrained Markov decision process formulation 

The tradeoff problem ( |4.4[ ) can be formulated as a constrained Markov decision problem (CMDP) 
[2|. The state space of the CMDP is the state space of the queue length, which is Z_|_ for the l-model 
and M+ for the R-model. The action spaces at each state g are the sets {0,..., min(g, S'^ax)} 
for the l-model and [0, mm(g, 5maa;)] foi' the R-model, both of which are compact for every g. 
The probabilistic evolution of the state of the CMDP from stage m to m -F 1 is given by (EH)- 
Associated with the CMDP there are two single stage costs: (i) the holding cost g at state g, and 
(ii) the service cost c(s) when an action s is taken at state g. 


122 






We redefine c(s) as the lower convex envelop^ of c{s),s G {0,..., Smax}- We note that the 
redefined cost function c(s) (c(s) : [0,S'maa:] —t 1^+) is a piecewise linear convex function. From 


and [27], it is possible to show that if Cc > c(A) then (4.4) has an optimal solution and there 
exists an optimal policy 7 G 

In the following, we show that for some values of Cc, there exists a stationary deterministic optimal 


policy for (4.4). Consider the following MDP: 


minimize [Qii, w) + PcACh, qo) - Cc)] • 

7Gr 


(4.5) 


We note that the above MDP has a single stage cost of {q, s) = q + j5c^c{s) in state q. From 
Ma et al. [3Bj, it is known that if there exists a fic^ > 0, such that any stationary deterministic 


optimal policy for (4.5), has an average service cost equal to the constraint Cc, then the same 


policy is optimal for the constrained problem (4.4). The factor can be interpreted as a Lagrange 


multiplier. The set of values of Cc for which such exist is denoted as as in Chapter 2. We 
note that the properties of any stationary deterministic optimal policy, which can be obtained from 
( |431 ), carry over to ( |4.4D if Cc G 

We note that the development in Altman |2l| which leads to Theorem 12.7, which shows that for 
every value of the constraint Cc, there exists a Lagrange multiplier for which there is a stationary 
deterministic policy which is optimal for both the unconstrained MDP and the CMDP, requires 
assumption (Bl) [2 Chapter 11], which does not hold for our model. 


In the next section, for the l-model we study (4.5) in detail. The properties of any stationary 


deterministic optimal policy are then used to motivate the definition of a class of admissible policies. 


The tradeoff problem (4.4) is then analysed for the class of admissible policies. We note that for 


Cc G there exists at least one optimal admissible policy. 


4.3 Asymptotic bounds for l-model 

4.3.1 An unconstrained MDP formulation 

The unconstrained MDP (|4j), which is obtained via the above Lagrange multiplier relaxation, is 
studied in |29j. However, we note that the development in |29j does not lead to an average cost 
optimality equation (ACOE). Since the ACOE enables us to obtain some additional properties of 
the optimal policy, in the next section we use the results from Sennott |!67| to show that there exists 
a stationary deterministic average cost optimal policy, which also satisfies an ACOE, for a single 
stage cost of C 0 {q, s) for our model. 


^We note that the lower convex envelope can be interpreted as the solution: c(s) = minimizeEc(X), such that 
EX = s. 
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The state space of the unconstrained MDP (4.5) is the state space Z+ of the queue length, which 
is countable. The action space at each state q is the set {0,..., min(( 7 , Smax)} which is compact. 
The probabilistic evolution of the state of the MDP from stage m to m + 1 is given by ( |4.lD . The 
single stage cost for the unconstrained MDP is c^(g, s) = q + (3c{s), where /3 > 0. We also assume 
that: 


A 2 : Pr{A[l] = a} > 0 , for all a e { 0 ,... ,Am.ax}, 

Lemma 4.3.1. If A < Smax, and if assumptions A1 and A2 hold, then there exists a stationary 
deterministic optimal policy 7 ^ for the unconstrained MDP, with optimal average cost satisfying 
the following ACOE: 

9 } + Jpiq)= ^ min 10/3(9, s) + EJ;3(g - s + A[l]) I, Vg > 0 , 

se{0,...,mm(q,Smax)} { J 

with 7 ^ using a batch size s^(g) at queue length q, satisfying 

s*^(q) = argmin | 0/3(9, s) + EJ/3(9 - s + A[l]) fvg > 0, 
se{0,...,min(q,Smaa:)} I J 

where J^(q) is the optimal relative value function. 


The proof is given in Appendix 4.A From now on, we assume A < Smax and that A1 and A2 hold. 
The following property of any optimal policy 7 ^ can also be obtained. 


01 : any stationary deterministic optimal policy 7 ^ is such that 5 ^( 9 ) is non-decreasing in 9 . 


The proof of the above property is similar to that of Theorem 3.2 (iii) of |29j, and is therefore 
omitted. We now state some observations which are obtained from the above lemma and 01. 


02 : The optimal average cost g*^ is independent of the initial state 90 and is finite. 

03 : For any policy, from assumptions A1 and A2, we note that from state 0 it is possible to reach 
any other state 9 . From 01, we obtain that any stationary deterministic optimal policy has a 
single recurrence class 'P'y*, of the form { 9 m,...}, where qm = min {9 : 3q' > q,Pqi^q > O}, 
and pqi^q = Pr {Q[m + 1] = 9 |Q[m] = 9 '} for the optimal policy under consideration. 


04 : We note that s*p{qm) = 0 by definition. From A2, we have that qm is an aperiodic state, and 
therefore the class of the Markov chain under 7 ^ is aperiodic. 


05 : From Lemma 


4.3.1 


for 7 ^, for 9 e 


we have that 


9 *p + -(9(9) — 0/3(9, 9^(9)) + IEJ/ 3(9 — 5^(9) -F A.[l]), 
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which verifies the drift condition (10.13) from [37|, with the Lyapunov function V(q) = J/siq), 
for a Markov chain restricted to We also note that cp{q, s*^{q)) is near-monotone |37] 
in q. Therefore using (37l Theorem 10.3], we obtain that any stationary deterministic optimal 
policy is Cy 3 -regular for a Markov chain on 'R-y*- Then the Markov chain under 7 ^ is positive 
recurrent on TZ^* with an associated invariant distribution. Furthermore the expected total 
cost of first passage from any state q G 1° another state q' G TZy* is finite (37] Theorem 
10.3]. 

Using property 05 of 7 ^ we prove the following lemma, which shows that any optimal policy is in 
fact non-idling. We note that if 7 ^ is non-idling, then = Z_|_ and the Markov chain under 7 ^ is 
irreducible. 

Lemma 4.3.2. Any stationary deterministic optimal policy 7 ^ is non-idling, i.e., s*^{q) > 0, for all 
q G TZy* and q > 0. 


The proof shows that if 7 ^ is such that there exists a positive qi G TZy* with s*^{qi) = 0, then 
it is possible to construct a history dependent non-stationary policy for which the average C/3(g, s) 
cost is strictly less, contradicting the optimality of 7 ^. The essential steps in the proof are: a) 
we consider a particular sample path of the queue evolution, for which it is assumed that at the 
start of a slot m, Q[m — 1] = (71 with S'[m] = 0, b) we obtain a new non-stationary policy which 
advances the service of one customeij^ from one of the succeeding slots to m while keeping the 
departure times of all other customers unchanged, and c) we show that for this new policy the total 
cp{q,s) cost decreases because i) the delay of the customer whose departure time was advanced 
has decreased and ii) convexity of the service cost function implies that the service cost at m, c(l), 
is less than or equal to the decrease in service cost at the slot where the customer was being served 
under policy 7 ^. Extension of the proof to the case of average cost, with the optimal policy being 


not irreducible, is more technical and is therefore presented in Appendix 4.B 


We recall that by solving the unconstrained MDP, we are able to get solutions to problem (4.4), 


with service cost constraint Cc, only if a Lagrange multiplier /3 exists such that the optimal policy 
for the unconstrained MDP with single stage cost C/ 3 (g, s) has an average service cost equal to Cc, 


i.e, if Cc G C>“. Thus in general, the properties 01, 02, 03, 04, and 05, as well as Lemma 4.3.2 


may not hold for all values of Cc in the original problem (4.4). 


4.3.2 The tradeoff problem 


As noted in Section 4.2.1 we consider problem ( ]4.4] ) for the set of randomized stationary policies T^, 
since there exists an optimal stationary policy. A policy 7 e r„ specifies the service batch size S{q) 


^Since qi is positive at least one customer will be present in the system at slot m, who will be served in some slot 

> m. 
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at a queue length q. We note that S{q) is a random variable with support on {0,, min(5maa;, q)}- 


We further restrict the study of problem (4.4) to the set of stationary admissible policies, whose 


definition is motivated by the properties 01, 02, 03, 04, 05, and Lemma [4.3.2[ We now define 
the notion of stability for a policy. 


Stability: A policy 7 e F* is said to be stable if: a) the Markov chain Q[m] under 7 is posi¬ 
tive recurrent with stationary distribution tt-j on the recurrence class corresponding to qo and b) 

Qh,Qo) < 00 . 


Admissibility: A policy 7 is called admissible if: 


G1 : it is stable, 

G2 : it induces an aperiodic, irreducible Markov chain Q[m], 

G3 : the average service rate at a queue length q, ES(g) is non-decreasing in q. 


The set of all admissible policies is Fq. We note that the above properties of admissible policies 


are motivated by the observations about stationary deterministic policies made in Section 4.3.1 


Property G1 is motivated by 02 and 05. Properties 03, 04 and Lemma 4.3.2 motivates property 


while G3 is motivated by 01. We note that the above definition of admissibility differs in the 
addition of property G3, from the definitions of admissible policies which were used by Berry and 
Gallager (71 Section IV] and Neely [43l Section III]. We also note that assumptions A1 and A2 have 
been used to motivate G2, through the properties 03 and 04, but as in Berry and Gallager (7||, we 
could assume G2. 


Remark 4.3.3. We now compare our definition of an admissible policy with that of Berry and 
Gallager ]7j. In |7|, it is required that a sequence of admissible policies form an ergodic Markov 
chain, i.e., an aperiodic, irreducible and positive recurrent Markov chain. Our admissible policies 
are also assumed to satisfy the same properties. In |7f, it is required that a sequence of admissible 
policies 7 fc are such that Qi'jk) < 00 and linifc^go Q{lk) = 00 . We also assume that Qijk) < 00 . 
However, we do not assume that limk^oo Qilk) = oo- In fact, we shall see that for a particular 
case (Case 1), the optimal sequence of policies 7 ^ is such that Q( 7 fc) < 00 . The third 

property in j7|, that admissible policies are assumed to satisfy is similar to our assumption Al. We 
note that the additional property G3 can be used to obtain additional insights about any stationary 
deterministic optimal policy. Furthermore, for Cc G there exists an admissible optimal policy. 


From G1 and G2, the average queue length as well as the average service cost are independent 
of the initial queue length since the Markov chain {Q[m]) is aperiodic, irreducible and stable |37]. 

^We note that the development of asymptotic lower bounds also holds under an assumption weaker than irre- 
ducibility. We can assume that under the policy 7, (i) there is only a single positive recurrent class R~f and (ii) the 
expected cumulative queue length and expected cumulative service cost starting from any state go until R-y is hit is 
finite. We note under assumption A2 and G3, R~f is a contiguous set. 
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The average queue length and average service cost for 7 G Tq are therefore denoted by < 5 ( 7 ) and 
0 ( 7 ). We note that < 5 ( 7 ) = Q and 0 ( 7 ) = c(5'(Q)), where Q denotes the stationary 


queue length. Since, in general, for every Cc, we do not know if Lemma [4.3.2| holds for the optimal 
constrained policy, admissible policies are not required to be non-idling. 


Objective : Our objective is to obtain the optimal tradeoff curve Q*{cc) while restricting our 
attention to the class of admissible policies Tq, where Q*{cc) is the optimal value of the following 
optimization problem 


TRADEOFF : minimize < 5 ( 7 ) such that C'( 7 ) < Cc- 

7sra 

We note that the TRADEOFF problem can be formulated for a larger class of policies, which are 
obtained by time sharing or mixing of policies jk £ Let Q*j^{cc) denote the optimal tradeoff 
curve, when we consider time shared policies also. We note that the tradeoff curve Q\j{cc) which 
is obtained from time sharing is the lower convex envelope of the points (cc,Q*(cc)). Since the 
asymptotic behaviour of Q^(cc) can be obtained from that of Q*{cc), as in Chapter 2, in the 
following we analyse Q*{cc) only. 

We note that given any e > 0, and for any Cc such that TRADEOFF is feasible, by definition there 
exists an admissible policy 7 such that < 5 ( 7 ) < Q*{cc) + e and C{'y) < Cc- Such a feasible admissible 
policy is called e-optimal in the following. 

We note that for any 7 G Tq, we have that E 7 r.^ 5 (Q) = A. We recall that c(s) was redefined as the 
piecewise linear lower convex envelope of c(s),s G {0,... ,Smax}- Then, from Jensen’s inequality, 
we have that for any policy 7 G Tq, C{'y) = E.,y^c{S{Q)) > c(A). Therefore, mf.ygra ^'( 7 ) > c(A). 
We note that TRADEOFF does not have any feasible solutions if Cc is less than c(A). 

We also note that c(A) is also the minimum average service cost which has to be expended for 
mean rate stability. In the following we show that c(A) = inf.ygra ^'( 7 ). We obtain an asymptotic 
characterization of Q*{cc) in the asymptotic regime as Cc i c(A) in the next section. 


4.3.3 Asymptotic analysis of TRADEOFF - Preliminaries 

The ideas used in the analysis of TRADEOFF are the same as those used for the analysis of 
INTERVAL-|uCHOICE-2 in Chapter 3. However, bounds on the stationary probability of the queue 
length for the DTMC {Q[m]) have to be developed, in order to relate the average service cost to 
the average queue length. As in Chapter 3, we identify three different cases based on the nature of 
the function c{s) at s = A. The cases are defined in terms of the quantities s; and Su defined as 
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follows: 


Su 


Si 


min {s : s G {[A] ,..., Smax — 1} , c(s + 1) — c(s) > c(s) — c(s — 1)} if this set is non-empty 
Smax Otherwise. 


max {s : s G {1,..., [AJ} , c(s -h 1) — c(s) > c{s) — c{s — 1 )} if this set is non-empty 
0 otherwise. 


We note that si and Su are analogous to the service rates /i; and defined in chapter 3. The 
three cases that we consider are: 


Case 1 : si = 0, si < X < Su 
Case 2 : s/ > 0, s; < A < 

Case 3 : s/ = A = 


We now show that c(A) can in fact be approached arbitrarily closely. 

Lemma 4.3.4. There exists a sequence of policies 7 ^^, G Ta such that c(A) can be approached 
arbitrarily closely, i.e., limfc^oo C'( 7 ej.) = c(A). Therefore c(A) = inf..),gra ^*( 7 ). 


The proof is given in Appendix 4.C We characterize the tradeoff curve Q*{cc) in the asymptotic 
regime where the cost constraint Cc approaches c(A). 


Similar to the definition of the line l{fi) in Chapter 3, here we define the line l{s) : [0, Smax] 1^+ 
as follows: 


1 . If Si < X < Su, then l{s) is the line through {si,c{si)) and {su,c{su))- 

2. If s; = A = Su, then l{s) is a line through (A, c(A)) with slope m chosen such that c(A) — 
c(A — 1) < m < c(A -l- 1) — c(A). 


The different cases along with the line l{s) are illustrated in Figure 4.1 We note that [c(5') — c(A)] = 
E^\ciS)-l{S)]. 


For a particular policy 7, if there is no source for confusion we use TT{q) to denote the stationary 
probability of queue length being q. The stationary probability of using a particular batch size 
s G {0,..., Smax} is denoted by 7r5(s). We now present two results which are used in the asymptotic 
characterization of Q*{cc). 


Lemma 4.3.5. For 7 G Ta, for some positive e < si, qs = ini {q :ES{q) > si — e} ,pd = 
= 0}, and p = 1 + ^, if Pr {Q < q^} (l +< ^, we have that 


Q(7) > 2 



1 

1 


2Pr{Q < 
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Figure 4.1: Illustration of the relationship between A, si, and Su along with the minimum average cost c(A) 
and the line l{s) for the three cases 


The proof is given in Appendix 4.D The above result is useful in obtaining an asymptotic lower 
bound to Q*{cc) as Cc i c(A), since in Cases 2 and 3, as Cc i c(A), for any sequence of feasible 
policies for TRADEOFF, Pr {Q < | 0. 


Remark 4.3.6. We note that a similar asymptotic lower bound has been derived in [44l Theorem 
2] (where admission control is allowed) and in [721 Theorem 2], where the assumption G3 has not 
been used. Although, the above result has been derived independently, we note that underlying 
all the three derivations, there is the idea of bounding the probability of an event by a particular 
sequence of transitions for a Markov chain, i.e., a sequence of transitions in which the state of the 
Markov chain becomes successively smaller. Furthermore, in our proof, using assumption G3, we 
obtain geometric bounds on the stationary probability of any queue length, which is not available 
in [44] as well as [ 72] . 


We note that in Chapter 3, since the queue length process Q{t) was a birth-death process, bounds 
on the stationary probability of the queue length could be obtained relatively easily. However, in this 
chapter, bounds on the stationary probability of discrete time Markov chains (DTMC) are required. 
In the following lemma, we present three bounds on the stationary probability of the queue length, 
one of which has been obtained by Bertsimas et al. [8J and [9] and the other two are state dependent 
extensions of the geometric bounds on the stationary probability of discrete time Markov chains 
presented in [8] and [9!|- 

Lemma 4.3.7. Let {Q[m],m > 0) be as in (EH), for an admissible policy 7. Let ea be as defined 
in assumption Al. Then, 

TAIL-PROB [ 18 ] : Suppose Vg > 0, E[Q[m-F 1] — (5[m]|(5[m] = q] > —d, where d is positive. 
Then for any finite qi and A; > 1 we have 

Pr{Q> qi + k} > f ) Pr{Q>qi}. 
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TAIL-PROB-STATE-DEP-1 : Suppose there exists a qd such that 


Vg G {0 ,... ,qd} ,E[Q[m + 1] - Q[m\\Q[m\ = g] > -d, 


where d is positive. Then for any qi, k > 0 such that 0 < gi + /c < we have 


Pr{Q>qi+k} > 


€„ + d 


Pr{Q > qi} 


+ 1- 


Pr{Q >qd + l} 


+ rf, 

1 

+ - TT{q)E[Q[m + l]-Q[m]\Q[m]=q] 

</=9d + l 

TAIL-PROB-STATE-DEP-2 : Suppose there exists a qd such that 

\/q G {0,.. . ,qd} ,E[Q[m + 1] - Q[m]\Q[m] = q] > -di, 

and Vg G + 1,...} ,E[Q[m + 1] - Q[m]\Q[m] = g] > -^ 2 , 
where d 2 > di > 0. Then for any qi, k >0 such that 0 < qi + k < qd, we have 


Pr {Q>qi+k] > 


+ di 


Pr{Q >qi} - [1- 


+ di 


d2 — di 
di 


Pr {Q>qd + l} 


The proof is presented in Appendix 4.E We now present the asymptotic characterization of Q*{c 
as Cc c(A). 


4.3.4 Asymptotic characterization of Q*{cc) as Cc i c(A) 


We first consider Case 2, the proof of the following asymptotic lower bound has already been briefly 

We use the geometric bound based lower bound on Q{'^) from Lemma 


discussed in Section 


4.1.1 


4.3.5 to obtain the asymptotic lower bound for Case 2. 


Lemma 4.3.8. For Case 2, given any sequence of admissible policies 7 fc with ( 7 ( 7 fc)-c(A) = 14 I 0, 
we have that Qi'yk) = ^ (log (t^))- 


Proof. Let us consider a particular policy 7 in the sequence jk with I 4 = V. From the definition of 
l{s) we have that [c(5(Q)) — 1{S{Q))] = V. From the convexity of c(s) and the linearity 

of l{s) we have 

00 

[c{ES{q))-l{ES{q))]<V. 

q=0 
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Now as c(s) > l{s), we have that 


93-1 


^ 7 r(g) {c{ES{q))-l{ES{q)))<V, 


9=0 


where qs, as in Lemma 4.3.5[ is inf {q : ES{q) > si — e} for a positive e < s;. We note that qs > 1. 
We note that for q < qs, c(ES{q)) — l{ES{q)) > me, where m is the tangent of the angle made by 
the line passing through (s; — l,c(si — 1)) and {si,c{si)) with l{s). Therefore we obtain that 

Pr{Q < qs} < —. 

me 


Using the above bound on Pr {Q < qs}, we have that for sufficiently small V, Pr {Q < gs} < g- 
Then, from Lemma [4.3.51 we have that 


<3(7) > 2 [logp 


me 

Wi 


- 1 


where the upper bound ^ on Pr {Q < ^ 5 } is used. Therefore for the sequence of policies 7 ^ with 
Vk I 0 , we have that 


Qi'Yk) = ( log ( — 


□ 


Remark 4.3.9. For Case 2, as for INTERVAL-/iCHOICE-2-2, we have a set of queue lengths Qh, 
which occur with high probability, such that ES{q) G G Qh- Let the drift in state q 

be A{q) = E [Q[m + 1] — Q[m]\Q[m] = q]. We note that, intuitively the drift A(g) for q € Qh 
is increasing and then decreasing for in Qh- Then we expect that the stationary distribution of 
the queue length is geometrically increasing and then decreasing as for INTERVAL-|uCHOICE-2-2. 
However, unlike INTERVAL-/iCHOICE-2-2, for the discrete time model we are only able to obtain a 
geometrically increasing upper bound. This bound suffices to obtain the Q ^log (xt)) asymptotic 
lower bound in the above lemma. 


We note that the model that is considered here is a simplified version of Neely's |433 model - there is 
only one queue evolving on Z+ and we have the fade state taking only a single value. The tradeoff 
optimal control algorithm (TOCA) of ^43j achieves the logarithmic tradeoff stated in the above 


lemma but for the problem (4.4), since TOCA is not admissible. Therefore, we propose a sequence 


of admissible policies that achieves the asymptotic logarithmic tradeoff of Q*{cc) in Lemma 4.3.8 


Lemma 4.3.10. Let a policy 7 be defined as follows. At a queue length q, 7 serves a batch size 
min(g, s(q)), where 

si, for 0 < g < qv, 


s{q) = 


Su, for qy < q. 
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where > 0. We obtain a sequence of policies 7 ^, by choosing = log where I 4 < 1 is 

a sequence decreasing to zero. Then for Case 2, 7 ^ is a sequence of admissible policies, such that 

C{ik)-c{\) = 0{Vk) and Q{^k) = O (log (^)). 


The proof of this lemma is given in Appendix|4.G[ We that the structure of the sequence of policies 


7 fc is similar to that in Lemma 2.3.12 We note that this proof is motivated by and borrows ideas 
from the derivation of the asymptotic upper bound for the sequence of TOCA policies in 
Corollary 2]. This leads to the following asymptotic characterization for Case 2. 


Proposition 4.3.11. For Case 2, the optimal tradeoff curve Q*{cc,k) = © (log ( 

Cc,k i c(A), for the sequence Cc,k = C{'yk), where 7 fc is the sequence of policies in Lemma 4.3.10 




Proof. For the sequence Cc,k = C{lk): we have that Q* iCc,k) < Q{lk) = o(log( 3 ^^^)). 
For e > 0 , consider any sequence of feasible e-optimal admissible policies 7 ^ for the sequence Cc fc. 
We have that < 3 ( 7 ^) = 0. (log ( c,fe-c(A) )) Qiik) ^ Q*{cc,k) + e- Therefore, Q*{cc,k) = 
^ (log Hence, for Cc,fe = C( 7 fc), we have that Q*(ce,fc) = 0 (log ( 3 -^^)). □ 

Remark 4.3.12. TOCA algorithm : We note that the set of all available power values (denoted 
by n in [43]) can be chosen such that the corresponding rates are {0,..., S'maa:}- As required in 
[43], the set is compact. The TOCA algorithm is parametrized by positive numbers w,e,q, and (3. 
The algorithm chooses at each slot m > 1, the batch size stoca such that 


STOCAi'm] = min argmin ^ (3c{s) — W[m]s'^,Q[m — 1]\ , 

\sC{0,...,5maa;} 


where 


W[m] = l{Q[m—l]>q} 

+2A:[m- 1] 

+ 

I{Q[m - 1] < q} 

_3 2X[to- 

1]' 


We note that stoca[t^] = 0 if lT[m] < 0. The sequence X[m],m > 0 is obtained from a virtual 
queue which evolves according to 

X[m + 1] = max(JT[m] - STOCAim + 1] + eI{Q[m] < q} ,0) + A[m + 1] + eI{Q[m] > q} . 

_ —e 

As in [43], let 5max = TAiaxiA^ax, Smax)- Let 0 < e < min(A - si,Su - X), w = , 

^max 

and q = ^log(/ 3 ). A sequence of policies 7^ is generated by choosing a sequence 13k = for a 
sequence I 4 i 0- Then from [431 Corollary 2], we have that 

Q{jk) = O (^log , C(7fc) = c(A) + O (Vk). 
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Therefore, we obtain that for the sequence of policies 7 ^, Qi'jk) = O ^log )-c(A) )) ' 

that fy[m] is a non-decreasing function of q, where Q[m — 1 ] = Since stocaIitt] is a non¬ 
decreasing function of fT[m], we have that STOCA[m] is a non-decreasing function of Q[m — 1]. 
However, we note that STOCA[m] is stationary only with respect to a state which includes an 
additional state variable X[m — 1] and hence is not admissible. Therefore, the above bound is an 


upper bound to the optimal value of (4.4) and not TRADEOFF. 


If for any subsequence Cc,k of such that there exists a constant 0 < m < 1 and a subsequence 

CTOCA,k of C( 7 fc) such that CTocA,k < Cc,fc and CTocA,k - c(A) > m{cc,k - c(A)), then we have 
that Q*icc,k) < Q{lk) = O (log (^j—= O (log (jr;^)) ■ 


We now obtain an asymptotic characterization for Case 3. The method used is the same as 
that summarized in Section 4.1.1| except that the geometric bound used is obtained from Lemma 


4.3.7 TAIL-PROB-STATE-DEP-1). Furthermore, we will see that in the asymptotic regime JR, the 
geometric bound reduces to a constant bound, which leads to the specific form of asymptotic lower 
bound for Case 3. 


Lemma 4.3.13. For Case 3, given any sequence of admissible policies 7 ^ with C{jk) — c(A) = 14 
0 , we have that Qi'jk) = (tr)- 


Proof. Let us consider a particular policy 7 in the sequence 7 ^ with I 4 = V. We define = 
sup{g : ES'(g) < A -F ey}, where ey > 0 will be chosen later. We note that as E5(0) = 0, the 
above set is non-empty. Suppose we assume that qd is finite. 

We note that by the admissibility of 7 , Vg G {0,..., g^}, ES'(g) < A -F ey. Hence, using d = ey. 


we have from Lemma 4.3.7 TAIL-PROB-STATE-DEP-1), for a g < g^ : 


Pr {Q > q + 1} > 


+ 1 - 


Ca + ey 


Ea + ey 

9-ti 


9+1 


Pr {Q > qd + l} + — 7r(g)E [Q[m-F 1] - (5[m]|Q[m] = g] 

ey ^^ 


<?=9d+l 


Or 


Pr {Q < q} < 1 — 


eg 

eg + ey y 

9+1 


1 - 


eg + ey 



E [Q[m -F 1] - Q[m]\Q[m] = q] ■K{q) |4,6) 


9=9<i+l 


as Pr{Q > gd + 1} > 0. For brevity, let = - (^ Y.'^=qa+i^ + 1] “ Q[m]\Q[m] = g] vr 

We note that Dt is positive, as for g > g^ -F 1, E[Q[m -F 1] — Q[m]\Q[m] = g] < — ey. Consider 
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the expression for Dt. We have that for q ^ + 1. 

E [Q[m] - Q[m + l]|Q[m] = q] = ^S{q) - A 

is positive as '^S{q) > A + ey- We note that by definition, c(s) is piecewise linear. Let m be 
the tangent of the angle between (i) the line passing through (A + 1, c(A + 1)) and (A, c(A)), and 
(ii) l{s). Then 7r{q) {ES{q) - A) < - l{ES{q))]. Furthermore 

from the convexity of c(.), linearity of l{.), and as c(s) — l{s) > 0, we have that 


7 r(g) [c{ES{q)) - l(ESiq))] < E^ [c(E5(Q)) - /(E5(Q))] < E [c(5(Q)) - 1{S{Q))] = V. 

<?=9d+l 


Therefore 


Now, as in the proof of Lemma 


Dt < 


V 

mey 


4.3.5 


we find a lower bound | on Q{'y) by finding the largest q 
such that Pr {Q < q} < ^. A lower bound qi to q can be obtained by using the upper bound ( |4.6[ ) 
on Pr{Q < q}. Let qi be the largest integer, if one exists, such that 


\^a + J 



] ( ^ X] IE[Q[m + l] -Q[m]|Q[m] =g]7r(g) 1 <-. 


<?=9d+l 


Then qi < q- Then we have to find qi such that 


1 - 


•• r-(- 


Ca + Cy 


f—) 


^qi+l\ I 

]Dt<^ 


1 + 2 A ^ 

or - < 

2 + 2 A “ 


f—) 

\^a + J 


2 ’ 
\ 91+1 


E^eyy^+' 2 + 2A 

or 1 H- < -— 

V J 1 + 2 A 

We note that if q^ = oo, then A = 0. However, qi satisfying the above inequality for finite qd is a 
lower bound for qi for qd = oo. Hence, we proceed with finding the above qi. Let q 2 be the largest 
integer such that 


J 1 + 2 A 


(4.7) 


Then q 2 < qi- From 0 and the upper bound on Dt, if gs is the largest integer such that 

\ ^aj 1 


V 

mev 
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then qs < q 2 . Or, we have that q^ is the largest integer such that 

We note that, as F i 0, if ^ oo, then the bound will be negative. We choose ey = aV, where 
a>^. Then we obtain that 


93 < log. 


1+^ I 1 + ^ 


- 1 , 


where the RHS is positive as F | 0. Therefore the maximum gs is at least 



/ 2 \ 



V + i-a) 

- 1 


Since Q( 7 ) and log = 0 {V), we have that for the sequence of 

policies 7 fc, ^( 7 ^) = O . □ 

Remark 4.3.14. As for INTERVAL-|uCHOICE-2-3, we note that there is a set of queue lengths Qh, 
which occur with high probability. Let the drift in state g be A(g) = E [Q[m + 1] — (5[m]|(5[m] = g]. 
Then Qh is the set of queue lengths such that A(g) — > 0, for g G as Cc | c(A). Intuitively, we 
expect that the stationary probabilities of all queue lengths in Qh are equal. With ey chosen to be 
aV as in the above proof, we have that 


Pr{Q <q] < 1 - 


aV 


Ca + aV 


q+P 


X a constant, 


g— X another constant. 

Ca 


This suggests that the constant stationary probability for queue lengths in Qh is 0{V), and therefore 
we obtain the LI (, 5 ;^ j asymptotic lower bound in the above lemma. 


Remark 4.3.15. We note that the sequence of randomized policies 7 ^^, in the proof of Lemma 4.3.4 
is such that 7 ^^, is admissible and C{^ek) — c (A) = €k and Q( 7 eJ = O (^) as I 0. 


This leads to the following asymptotic characterization of case 3. The proof is similar to that of 
Proposition 4.3. 11| 


Proposition 4.3.16. For Case 3, the optimal tradeoff curve Q*{cc^k) = © k-c{x) ) i ^(A), 

for the sequence Cc,k = 


Case 1: We note that in Case 1, Q*{cc) does not grow to infinity as Cc approaches c(A). In 
fact, the policy 7 ^, which serves S[m + 1] = min((5[m], s^j), has the finite minimum average 
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queue length over all admissible policies which achieve an average service cost of c(A). First of 
all, we note that any admissible policy 7 which has ^(7) = c(A), will have Pr{S{q) > = 0, 

Vg and Pr {S[m] > = 0,Vm > 1. For a given realization of the arrival process and the 

randomization of the batch sizes, let q*[m] and q[m] be the evolution of the queue process under 
7„ and 7 respectively. Then we note that q*[m] < g[m],Vm, and therefore 7^ has the least average 
queue length over all policies which have their average service cost equal to c(A). Furthermore, 
Q*{cc) < Q{lu) for Cc > c(A). 

For Case 1, we are only able to obtain a tight lower bound for a restricted case. We show that if 
Su = 1, then for any sequence of non-idling deterministic 7 ^ G Fa, for which C( 7 fc) — c(A) = f4, 
we have that Q{^k) = + I + ^ (f 41 og (\|:))- We note that if Su = 1, then Qi'ju) = 

+ 7 ’ f''°'^ IBJ- Thus as y I 0, we have that the asymptotic lower bound has Qiju) as the 
limit point. Furthermore, we note that the asymptotic order matches with what that was derived 
for FINITE-/iCFIOICE-l in Chapter 2. 

We present the lower bound on the average queue length in a series of steps. Consider a particular 
policy 7 in the above sequence in the sequence jk, with C'(7) — c(A) = V. 

Let qu = sup{(7 : s(q) < s^}. Since we have restricted attention to non-idling deterministic 7 and 
Su = 1 , we have that s(q) = = 1 for (7 G In the following lemma, we obtain an 

upper bound on Pr {Q > qu}, which will be used to obtain an upper bound on qu- 

We note that from Assumption Al, Amax > Smax- Therefore starting from any queue length q, 
the queue length in the next slot is at most q — Amax or q + Amax- 


Lemma 4.3.17. For any non-idling deterministic 7 G F^, if qu > Amax, then for any A: > 0, 

Pr {Q>qu + k}< min , 1 , 


where pu = 


The proof is discussed in Appendix |4. FI The proof is very similar to that of Lemma 1 of Bertsimas 
[8j and |9j but with some slight modification. We note that c(s) < ci(s), where 


ci(s) 


c(s), 

C(S„) + (s - Su) 

K ^ ' ‘^max ^ ' 


for s G [0,s„], 
for S G {Su, Smax]- 


136 









We note that ci(s) = l{s) for s G [0, s„]. Then, 


V = C(j) - c(A) < [ci(s(Q)) - /(s(Q))], and, 

E,r [ci(s(Q)) - Hs(<3))] = ^ [ciis{q)) - l{s{q))], and, 

q>qu 

Y,T^iq)[cMq))-lisiq))] < ^ 7r{q) [m{Smax - s«)] , 
q>qu q>qu 

where m is the tangent of the angle between ci(s) and l{s) at Su- Therefore, we have that 


V < ni{Smax Su) E 

q>qu 

Pr {Q > qu} > ^ 


or, 


(4.8) 


'^{Smax Su) 

We now proceed to find a upper bound on qu by combining the above lower bound on Pr {Q > qu} 


with the upper bound derived in Lemma 4.3.17 


We note that if qu < Amax, then Amax is an upper bound on qu- Suppose qu > Amax, then from 


Lemma 4.3.17 and (4.8), we have that 

F 


( Smax Su) 

Since pu < 1, we have that 


r__1 qu 

<Pr{Q> qu} < 


1 \ 2 Amax m{Smax — Su) 

— < .. or, 

PuJ V 


qu < 271 max log_L 


^ ( Pmax Su) 

V 


(4.9) 


Lemma 4.3.18. The average queue length for the policy 7, 

A Smax Su 


Q{i) > 


a 


+ 77 - 


^{su A) 2 Su A 


quPr {Q > qu} + '^ Pr {Q > q} 


q=qu 


Proof. We note that the policy 7 is admissible. Squaring both sides of the evolution equation ( |4.lD , 
taking expectations with respect to the stationary distribution, and simplifying we obtain that 


2E [Q{S - A)] = E^2 + ES^ - 2E^ES’. 

We note that Eyl^ — Ej4E5 = a^. Since Su = 1, we have that ES"^ = ^s('S)s^ > 
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suEfrr ^s(s)s = As. 


2EQ(S-A) > + A(sn-A), or, 


a 


^7r(g)g(E5(g) - A) > ^ + A(s„-A), 

5=0 


Or, we have that 


(4.10) 


^Tr{q)q{su - A) + ^ Tr{q)q{Smax - Su) > ^ + A(s„-A), 


5=0 


5=<?u + l 


(Su - ^) '^{q)Q > Y + HSu - A) - {Smax “ ^n) ^ 7r(g)g 


5=0 


5=5u+l 


(4.11) 


Simplifying the term vr(g)g, we have 


Y1 = '^{qu + q)'n-{qu + q), 

5=<?ii + l 5=1 


= qu E -w+E qTr(qu + q ). 

5=911 + 1 5=1 


(4.12) 


Then from (4.11) we have that 


<3(7) > 


j^frA) + ^ - a'" *“5 > "J + S/" <'3 > 


□ 


We now use the upper bound on q^ from (4.9) and the upper bound on Pr {Q > qu + k} from 
Lemma 


4.3.17 


in the above lower bound on Q{'y) to obtain our final result. 


Lemma 4.3.19. If = 1, then for any sequence of non-idling, deterministic 7 ^ G T^ such that 

C{7k) - c(A) = 14 I 0, we have that Qi'yk) = 2{.s'f-\) + t (Vfclog (^))- 


Proof. We consider a 7 in the sequence 7 ^, with C{'y) — c(A) = V. Then, from Lemma 
have that 


4.3.18 


we 


Qii) > 


^ ^ Smax 

2 ( 5 -^ — A) 2 Su — \ 


[quPr{Q > w} + X] > ^}) • 


q=qu 
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Consider Pr {Q > q}. We have that 


2 ^max 1 


J2Pr{Q>q}=Yl 12 Pr {Q > Qu + TTilAfYiax P P\ ■ 


q=qn 


m=0 k=0 


From the proof of Lemma 4.3.17 we have that 


r m2Amax-^l^ ~\ 

Pr {Q > QuP mlArnax + k} < pi 'Pr{Q> q^} , 

r m2A77i,ax ~[ 

< pi 'Pr{Q> g4 , since pu < 1, 


= Pr{Q>qu}p: 


Substituting in (4.13), we have that 


^ Pr {Q > g} < 2AmaxPr {Q > qu} ^ Pu: 

q=qu m=0 

— ‘2‘-AmaxP‘^ {Q ^ Qu} Z • 

1 - Pn 


Now we obtain an upper bound on Pr {Q > q^}. We note that 


Then 


c{l) - c(A) = [c(s(9)) - l{s{q))] ■ 

q>qu 


mu ^ vr(g) [s{q) - sj < (^( 7 ) - c(A) = V, 
q>qu 

where ruu is the tangent of the angle made by the line through {c{su + 1 ), Su + 1 ) and (. 
with l{s). We note that for q > qu, s{q) — Su> 1- Therefore 

V 7r{q) = Pr{Q > qu} < —. 

mu 


Using the above upper bound in (4.14), we obtain that 


Pr [Q > q} < 2Au 


V 


q=qu 


/1 A * 

muil - Pu) 


From (4.12), we also obtain that 


quPr {Q > qu] < log j_ 

TTlii Pu 


m(^Sfnax Su) 

V 


(4.13) 


(4.14) 


(^u) ,Su) 

(4.15) 
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Then we have that 


7 ^/ \ ^ ^ ^ Smax , 

Q{i) > -( -^ +0-^ -log^ 


n^iS-max Su) 


V 


+ 2j4^ 


V 


' / -f \ 

niuil - Pu) 


Thus, for the sequence of policies 7 ^, with Vk i 0, we obtain that 

^2 A „ . r 1 


Qiik) = 


a 


2 (.„-A) + 2-°7‘‘°H7 


□ 


Remark 4.3.20. We note that the above asymptotic lower bounds can be obtained for admissible 
policies, even for general holding costs. Suppose, the holding cost is h{q) in state q, instead of the 
queue length q. We assume that /i(g) is a strictly increasing function of q. Then, instead of Q{pi) 
for an admissible policy, we are interested in the average holding cost Hipi) = '^^o'^{q)h{q). 
We assume that for admissible policies H{'y) is finite. We note that asymptotic lower bounds on 
H{'y) can be obtained quite easily, from the above results. Consider the random variable h{Q) for 
a policy 7. Then we obtain a lower bound | on H{j), where h is the largest number such that 
Pr {h{Q) < fi} < 2 ' If inverse function h~^ of h exists, then we have that h is the largest 
number such that Pr {Q < We note that we have already obtained lower bounds 

qi to q, where q is the largest integer such that Pr {Q < g)} < Therefore, we obtain that 
P(7) > 


4.3.5 Asymptotic characterization of admissible policies for TRADEOFF 

We consider a sequence of Cc,fc i c(A) for TRADEOFF. Let 7 ^ be any sequence of feasible policies 
for the sequence Cc,k- In this section, we obtain an asymptotic characterization of 7 ^. Our approach 


is similar to that in Sections 2.3.3 and 3.2.3 for the state dependent M/M/1 model. However, we 


are unable to obtain asymptotic upper bounds. Since, only the asymptotic upper bounds depended 
on the order-optimality property, the bounds that we derive here hold for any sequence of feasible 
policies for the sequence Cc,k- 

We first obtain the bounds Pi {A} and Pu{A} as in Section [3.2.3 We note that the elements 
of sets A are average service rates ES{q). Let A C [ 0 , 5 ; — ev]|J[s„ -L ev,Smax]- Let Qa = 
{q : ES{q) G Ay}- Proceeding as in the proof of Lemma 4.3.8| we have that Pr {Q G Qa} < 


Pu{Qa) = 7 ^- Then, we have that 


Pu{Qa) 




1 

ma ’ 


if 


if ey = 0 < (5 < 1 , 

ey = aV. 


(4.16) 


Let A = [si — ey,Su + ey]. Let Qa be defined as before. Then using (4.16), we have that 
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Pr{Q e Qa} > Pi{Qa), where 


Pi{Qa) = ma^'^^v aV,0<6<l, 


(4.17) 


if ey = ay. 

We note that the above two bounds hold for cases 2 and 3. 

Consider the sets Ai = [ 0 ,sz-ey], A2 = (sz-ey,s], ^3 = (s,s„ + ey], and A4 = [su +ey, Smax]- 


From (4.16) we have that 


V 


7r(A2) + ttIAs) = 1 - 7r(Ai) - 7r(A4) > 1-^—, and 

mev 

(s; - ey)7r(A2) + S7r(^3) < A. 


We have that 


7r(^3) < 


T^{A2) + 


ev)TT{A2) 

s 


> 


X- {si- ev)7r{A2) 
s 


1 - 


V 

mev 


and, 


Therefore, 


T^{A2) 
-^{M) > — 

s - 


' s - {si - ey) 1 ^ g - A _ V 

s J ~ g mev ’ 

g - A gy 

(si-ev) mev {s - {si - ev))' 


The above lower bound can be non-negative only if g > A. In the following, we set g = A -F e'y, 
where X + e'y is assumed to be less than or equal to Su + ey and A-Fey for cases 2 and 3 respectively. 
Then we have that. 


T^{A2) > ^ 


-V 


Si + Cy + ey 


gy 

mev (A - s; -F e'y + ey) 


(4.18) 


We have the following result. 


Lemma 4.3.21. For any sequence of admissible policies 7 ^, with C{'yk) — c(A) = I 4 0, and 
Qa = {q ■ '^S{q) G A} for a ^ C [0, Smax], we have that 


IQaI 


fl ^log ) f°'' Case 2, if ^ = [si — aV^, Su + aV^], 0 < <5 < 1, a > 0, 

< Q ^log ) f°'' Case 2, if ^ = [s/ — aV^, A -F e], 0 < <5 < 1, a > 0, e > 0, 

fl : foi' Case 3, if ^4 = [0, A -F aV],a > 0. 


Proof. Consider a particular policy 7 in the above sequence with C{'y) — c(A) 


V. Let us first 
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consider the case where A = [si — ey, Su + ey]. From (4.17), we have that Pi{Qa) > 1 — , by 

choosing ey = aV^, 0 < 5 < 1. 

We proceed as follows for Case 2. Let qg = sup{g : ES{q) < si — ey}. Then, we have that 
Pr {Q < qs} < 




-. Suppose qi is the largest integer such that 

It T/1-'5 

Pr{Q <qs}'^ — <1 -, 

^ Pd ma 


where p and pd are as in Lemma 


4.3.5 


then qi < \Qav\- We then obtain that qi = Q (log (y))■ 


Now we consider A of the form [si — ey, s], for s = A + e'y < Sm + ey as in ( 4. IS] ). For this A, 
since the asymptotic lower bound on \Qa\ is (^) and is obtained as in the previous case, we do 
not present the derivation here. For Case 2, we choose ey = aW^,0 < <5 < 1, and eC = e, where 


e > 0 is such that A + e < Then from (4.18), we have that 

, ,, e' 

7 r{A) > 


X — Si + e' + aV^ m (A — s; + e + aV^)' 


(4.19) 


Now proceeding as for the case when A = [s; — ev,Su + ey] above, we obtain that for A = 
[si - ey, A + e], \Qa\ = ^ (log (y)). 

We now consider Case 3. For Case 3, we consider A to be [0,s„ + ey] rather than [si — 
^v,Su + ey]. We also choose ey to be aV. Then, we have that Pi{Qa) > 1 — Let 
qd = sup{g : ^S{q) < Su + ey}- We note that Pr {Q < qd — 1} = 1 — Pr {Q > qd}. From 


TAIL-PROB-STATE-DEP-2 Lemma 4.3.7 we have that 


Pr {Q < qd - 'i-} < 1 - 


+ ey 


Qd 


- 1 - 


fa + ey 


Qd 


— -®^'S'(g)]7r(g) 

ey —,, 


<?=9d+l 


Then proceeding as in the proof of Lemma 


4.3.13 


we have that qd = ^ (y). Therefore, |Qyi| = 

□ 


4.3.6 An asymptotic lower bound for the tradeoff problem (4.4) 


In this section, using the asymptotic results for TRADEOFF, derived in Section |4.3.4[ we derive 
lower bounds for the optimal value of ( |4.4[ ) for a set of Cc, via the Lagrange dual of ( |4.4[ ). We note 
that the same approach applies to the tradeoff problems in Chapter 5. The asymptotic results for 
(|4.4[) are derived only for cases 2 and 3. 


For (|4.4D, from Section 4.2.1 we have that there exists a stationary optimal policy. Therefore, the 
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optimal value of (4.4) is equal to that of 


minimize Qil), 

such that C{'y) < Cc- (4.20) 

However, in this chapter, the problem that we have considered is 

minimize Qi^y), 

■y&Ta 

such that C{'y) < Cc- (4-21) 


We note that the optimal value of (4.20) is lower bounded by the optimal value of its Lagrange 
dual: 


max 

/3>0 


min [Q( 7 ) + /3(C(7)-Cc)] 

s 


(4.22) 


4.3.1 


we have that for any /? > 0 , there exists an admissible policy 7 ^ which achieves 


From Lemma 

the minimum for the problem min.^grs [Q( 7 ) + P (^"( 7 ) — Cc)]. In the following, we show that an 


asymptotic lower bound to the solution of (|4.20[) can be obtained using (4.22) from the asymptotic 


behaviour of the optimal solution of (4.21), for certain sequences of Cc as Cc i c(A). 


We note that for any sequence /3 f 00 , it can be shown that C{'y'^) | c(A). For the following 
analysis, we also consider the MDP: 


min [g( 7 ) + /3 (^( 7 ) - c(A))] . 

7^-1- s 


(4.23) 


We note that for a /3 > 0, 7 ^ is optimal for both (4.22) and (4.23). We have the following result. 


Proposition 4.3.22. Suppose "fk is any sequence of policies such that C{'^k) — c(A) = I 4 | 0 and 


Q{ik) 


/ 

^ (w)) ’ 

O ) foi' Case 3. 


Let Cc^k be any sequence such that 


0 (^( 7 *;) — c(A)) , for Case 2, 

0 ((C'(7fc) - c(A))^^ , for Case 3. 
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Then, for the tradeoff problem (4.20), for the sequence Cc,k i c(A) we have that 


Q*(Cc,fc) = < 




n 




, for Case 3. 


Proof. We consider Case 2 first. Let 7 ^ be any sequence of policies (admissible or otherwise), 
which is such that C{'yk) — c(A) = | 0 and Q^'jk) = O ^log (^tr))' Lemma 


4.3.10 


we note that at least one such sequence exists. Let Pk = )-c(A)) ' show that (^( 7 ^ ) — 

c(A) = C) where 0 < 5 < 1. We proceed by contradiction. Suppose (^( 7 ^ ) — c(A) is 


not O [ ^ j. Then C(j^ ) — c(A) is w ^ ^ j. Therefore, the optimal value of (4.23), Qij^ ) + 




- ) — c 


= UJ 


l-<5 


Q{7k) + h{C{lk)-c{X)) = 0 


. However, we note that the sequence of policies 7 ^ is such that 
, which contradicts the optimality of the sequence 7*- . 

Pk 


Therefore, 0 ( 7 !J - c(A) = O (^ ^ ). 


Consider a sequence of Cc,a: i c(A) for (4.20). Suppose Cc,a; is such that Cc,fc— c(A) = 0 {C{ 7 k) — c(A)). 


Then (4.22) can be bounded below as 


max min [Qi'j) + /3 (<7(7) - Cc,k)] > + h (pilD “ c 


/ 3 >o 7 ers 


We have that 13k 


Pk (Cc,A; c(A)) 


Pk 


— C 


> 0. Since Cc^k — c(A) = O {C{7k) — c(A)), we have that 


Pk {cc,k — c(A)) = 0(1). Furthermore, since 0 ( 7 ^ ) — c(A) = O ^ j, we have that Q{7*^ ) = 


n 


. Since Cc,k — c(A) is also Q (C( 7 k) — c(A)) we have that 


+ Pk [C{7i) - c(A) ) - Pk ic.,k - C(A)) = fl ^log 


1 


which provides an asymptotic lower bound for (4.20). 

We now consider Case 3. Let 7 ^ be any sequence of policies, which is such that C{ 7 k) — c(A) = 


14 4 , 0 and Q{ 7 k) = O ■ From Lemma 
Let Pk = 


4.3.4 


(C(7fe)-c(A)) 


we note that at least one such sequence exists. 

We show that C{ 7 *- ) — c(A) = 0\ ). We proceed by assuming that 

P'= \VPk J 

. Then we have that the optimal 


C( 7 j J - c(A) is not O ( m y Then ) - c( A) is ^ ^ ^ 


value of (4.23), Q{ 7 *^ ) + Pk “ '^(A)^ = ^ sequence of policies 

7 fe is such that Q{ 7 k) + Pk{C{ 7 k) — c(A)) = O fvYfcY which contradicts the optimality of the 
sequence 7 *^ . Therefore, C{ 7 \ ) — c(A) = O 


Pk^ 


y/Pk J 
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Then, proceeding as in Case 2, for any sequence of Cc,a: such that Cc,k—c{X) = 0 
we have that for Case 3, the optimal value of 


(4.20) is Q 


t.—cfA) 



□ 


We note that since for G there exists an admissible optimal policy for (4.4), the asymptotic 
lower bounds obtained in Lemma 4.3.8 and Lemma 4.3. 13| apply directly. The above Lagrange 
dual approach shows that the asymptotic lower bounds also apply to (4.4) for the sequences Cc,fc 
considered above. Since the sequence of admissible optimal policies for the sequence c'^ ^ satisfy the 
properties required for 7 ^ stated in the above proposition, we note that the set of Cc,k for which the 
above lower bound holds also contains C>“. However, we are unable to show that for any sequence 
Cc,k i c(A) the asymptotic lower bounds in the above proposition hold. 


4.3.7 Asymptotic lower bounds for ergodic {A[m],m > 1 ) 

We note that when the arrival process {A[m],m > 1 ) is an ergodic batch arrival process, the 
optimal policy for the tradeoff problem ( |4.4D may not be stationary. But in this section, we consider 
the set of policies T^, which are such that the batch size S{q) used for service in slot m is a 
function only of the queue length Q[m — l] = q, and is independent of anything else. We note that 
S{Q[m — 1]) could be a randomized function of Q[m — 1]. The asymptotic lower bound presented 
here is significant, in that it complements the asymptotic upper bound obtained for Markov batch 
arrival processes in (451 Section 4.9] and |[30]. 

We assume that (A[m]) is ergodic, so that almost surely 

1 ^ 

lim — V Aim] = EAfll = A, 

M^oo M 

m=l 

and A < Smax, where Smax is the largest batch size which can be served, as defined before. We 
also assume that the arrival process {A[m]) is such that 

NAl : Let a[m- 1] = ((5[0] = = ai,Q[l] = qi,A[2] = a 2 , ■ ■ ■, A[m - 1] = am-i,Q[m - 

1] = qm-i)- We assume that 

inf min Pr {^[m] = Q\a\m — 1]} = t'a > 0. 

We restrict to policies 7 G T^ for which the following limits exist 

\\m Pr {Q[m] = q\Q[ff\ = qo} = 7 r(g'), Vg'G Z+, (4.24) 

m^oo 

'"'tf' ~ f- policies 7 G T^ for which the above limits exist, also have 
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well defined 7rs(s), Vs G {0,..., Smax}, where 


1 ^ 

7 r,(s) = lim — Pr{S[m] = s|Q[0] = . 

M^OO M 

m=l 


(4.25) 


For such a policy 7, the average service cost is C{'y) = '^s{s)c{s) and the average queue 

length is Qij) is defined as 

To obtain an asymptotic characterization of the tradeoff, we again restrict to a class Tq of admissible 
policies. However, we use a weaker definitiorj^ of admissible policies, compared with the definition 


in Section 4.3.2 A policy 7 G is admissible if: 


NGl : the limits in (4.24) exist for 7, 


NG2 : 7 is mean rate stable (see [45j), i.e., Y2s=o^ '^s{s)s = A, 

NG3 : the average service rate E5(q) is a non-decreasing function of q for 7. 


Then for any admissible policy 7, we have that C{'y) > c(A), as before, by applying Jensen's 
inequality. 

Let Uv be a random variable with support on {0,..., Smax} and KUy = A-|-f^. Let {Uv[m],m > 1 ) 
be an IID sequence with Uv[m] ~ Uy- Consider a particular policy 'yy for a ^ > 0, which 
chooses S[m] = mm{Q[m — l],Uy[m]). Using PH Lemma 1] we have that the limit Tr{q) = 
\mim-^oo Pr {Q[m] = q} exists. We note that then the evolution of the queue can be written as 

Q[m] = max ^Q[m — 1] — Uy[m],0^ + A[m], for m > 1, with (5[0] = qo- 

Let Q'[l] = max(( 7 o — and 

Q'[m] = max (^'[m — 1] -F A[m — 1] — , for m > 2. 

We note that Q[m] = Q'[m] + A[m],m > 1 . Since the sequence of random variables C[m] = 
A[m] — Uy[m + l],m > 1 is ergodic and EC[1] = A — (A -F fV) < 0, from [121 Chapter 1, Theorem 
7] we have that lirnm^-oo Pr{Q'[m] < 00 } = 1 and lirnm-^-oo Pr{Q'[m] = q} = P{q) exists. Since 
^[1] < Amax we have that Ymim^oo Pr {Q[m] < 00 } = 1. Thus 7 y satisfies properties NGl and 
NG2. We also note that by construction 7 y satisfies property NG3 and hence 7 y is admissible. We 
note that C{'yy) < Ec([/y [1]). Now consider the sequence of admissible policies 7 y for a sequence 


^The set of admissible policies in this section contains the set of admissible policies defined in the previous sections. 
This shows that the development of the log (y) asymptotic lower bound can be obtained under weaker assumptions 
than what was assumed in the previous sections. 
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y I 0. Then it is always possible to choose|^the distribution of Uy such that C{'yv) = c{X) + 0{V) 
as y I 0. Therefore, as before, c(A) = inf^ygra C{'j). 


We note that c(A) is piecewise linear and three cases arise depending on the value of A, as shown 


in Section 4.3.3 We define si, Su, and the line l{s) as in Section 4.3.3 For Case 1, we do not have 
a tight asymptotic lower bound. We now obtain an asymptotic lower bound which applies to Cases 


2 and 3. We note that the bound is obtained by generalizing the proofs of Lemmas 4.3.5 and 4.3.8 


We also note that this asymptotic lower bound holds even if (j4[m]) is just stationary rather than 
ergodic. Ergodicity of {A[m]) was required for proving that c(A) = inf..),gra C{'y). 

For some positive e < si, let Qs = inf {q : E5(g) > s; — e}. We note that the proof of the following 
asymptotic lower bound also follows the methodology summarized in Section 4.1.1[ except that the 
geometric bound on the stationary probability distribution for the ergodic process Q[m] is obtained 
from the assumptions NAl and NG3. 


Lemma 4.3.23. For an ergodic arrival process {A[m], m > 1), satisfying NAl, and for any sequence 
of admissible policies 7 ^ (satisfying NGl, NG2, and NG3), with C'( 7 fc) — c(A) = | 0, we have 

that Qi'jk) = ^ (^log Cases 2 and 3. 


Proof. For a particular policy 7 in the above sequence with I 4 = as in the proof of Lemma 
14.3.81 we have that 




ciES{q))-liESiq)) 


< V, 


since c(s) is convex and l{s) is linear. Therefore, 


93-1 


9=0 


< 


V 

mie’ 


where mi is the tangent of the angle between (i) the line passing through (si — l,c(si — 1)) and 
(si,c(si)) and (ii) l(s). 


As in the proof of Lemma 


4.3.5 


we have that Fr{S(q) > 0} > 4^— > qg, since for q > qg, 


ES{q) > Si — e. Now we relate the stationary probability 7r{q),q > qg to Ylq=o^ 
that for a (7 > qg and for every m > 0 


Pr {Q[m + 1] < q\Q[0] = qo} = Pr {Q[m] - S{Q[m]) + A[m + 1] < q\Q[0] = qo} , 


®For small enough V, a distribution for Uv that gives mass to either (i) A and A + 1, if A is an integer or (ii) [AJ 
and [A], if A is not an integer, can be chosen such that ¥,Uv — X + V and Ec{Uv) = c(A) + mV, where m is the 
slope of the line joining (A, c(A)) and (A + 1, c(A + 1)) for (i) or the line Joining ([Aj , c([AJ)) and ([A] , c([A])) for 
(ii). 


147 

















which can be written as 


IEs[i],Q[i],...,Q[m-i],SH Pr\Q[m] - S{Q[m]) + A[m + 1] < g 


Q[0] = qo,S[l],Q[l],...,Q[m- l],S[m] 


which is 


> IE5[i],Q[i],...,Q[m-i],SH Pr{Q[m] = g|(5[0] = go,-Sfl], Q[l],..., Q[m - 1], 5[m]} x 


Pr{S{Q[m]) > 0|(3[0] = qo, S[1],Q[1],... ,Q[m - 1], S[m],Q[m] = q} x 

Pr {A[m + 1] = 0|(5[0] = go, 5'[1], <3[1], ■ ■ ■, QVn - l],S[m],Q[m] = q, {S{Q[m]) > 0}} 


We note that the batch size S{Q[m]) is chosen independently of the history of the queue length 
evolution. We also note that the evolution ((^[O] = go, 5[1], Q[l], ..., Q[m — 1], S[m], Q[m] = q) is 
equivalent to the evolution cj[m] = {q^, A[1],Q[1], A[2],Q[2],..., A[m],Q[m] = g). Furthermore, 
A[m + 1] is independent of the batch size S'[m + 1] = S{Q[m]) given Q[m]. Therefore, using NAl, 
the above lower bound can be written as 



Pr {S((5[m]) > 0|(5[m] = g} x Pr {A[m + 1] = 0|(T[m]} . 

Then using the property NAl and the above lower bound on Pr{S(g) > 0} for g > g^ we have 
that Pr {Q[m + 1] < g|Q[0] = go} 



= Pr {Q[m] = q\Q[0] = go} ^ — -u, 

^max 


'max 


Defining pd = ^ ^ Va and p = 1 + ^, and proceeding as in the proof of Lemma 4.3.5 we have 

' Omax ' Pd _ 


that for any non-negative k, q = qs + k, and for every m > 0, 


Pr {Q[m] = q\Q[0] = go} < Pr {Q[m -F 1] < gs|(5[0] = go} —• 


Pd 


Therefore, 


1 

M 


M-l 



rC 1 ^ 

^ mYI Pr{Q['>TT] < 9^I<3[o] = w} + 

ra. L n 


^ Pr {Q[m] = g|Q[0] = go} < 


m=0 


m=0 


Pr{Q[M] < g,|Q[0] = go} - Pr{Q[0] < g,|Q[0] = go} 


M 
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Then as M —oo, since 7 is admissible, we have that 


vr(g) < — X] 

g=0 


Proceeding similarly as in the proof of Lemma 4.3.5 (from (4.31)), we can show that 


Qii) > 2 


log. 


2Er=o'^(5), 



Since Yl'I/Lo '^(q) — 1^°'' sequence of policies 7 ^, Qi^k) = 12 ^log 


Tf . □ 


We note that the above asymptotic lower bound is not tight for Case 3 for I ID {A[m\) since we 
have a fl (tt) bound on Q^’jk)- 

In [43 Theorem 4.12] and [30], it has been shown that if {A[m],m > 1) is Markov, i.e., 

Pr{A[m + 1] = am+i\A[m] = a-m, ■ ■ -,^[1] = ai} = Pr {A[m + 1] = am+i\A[m] = am} , 

then for a sequence of Quadratic Lyapunov Algorithm (QLA) policies, parametrized by a sequence 
14 i 0. the average queue length is O for an average service cost I 4 more than c(A). 

We note that the QLA algorithm chooses a deterministic batch size s{q) for service based on the 
current queue length only (unlike, say TOCA, for which the batch size is chosen as a function of 
other auxiliary variables also). Therefore, the QLA algorithm falls in the restricted class of admissible 
policies considered in this section. We note that the batch sizes are chosen deterministically, 
therefore we have that 

Pr{A[m + 1] = am+i\A[m] = am, ■ ■ ■,A[l] = ai,Q[0] = go} = 

Pr{A[m + 1] = am+i\Q[m] = qm,A[m] = am,Q[m - 1] = qm-i, ■ ■ • ,^[1] = ai,(5[0] = go} , 

for the given policy, for (gn,0 < n < m), such that g^+i = q-n — s{qn) + a^+i. Then we note that 
the above asymptotic lower bound applies, for the QLA algorithm, under the assumption 

Pr{A[m] = am\A[l] = ai, A[2] = 02 , •. .,A[m - 1] = am-i,Q[0] = go} = 

Pr{A[m\ = am\A[l] = ai, A[2] = 02 , •. .,A[m - 1] = a^-i} • 
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4.4 Asymptotic bounds for R-model 


As for the integer valued case, the tradeoff problem is to obtain Q*{cc,qo), which is the optimal 
value of 

minimize Q( 7 , go) such that C{'y,qo) < Cc, 

7er 

where Cc > 0 is the average service cost constraint. 


We recall that the above problem can again be formulated as a constrained Markov decision problem 
over the class of policies F. However, since the R-model is usually used as an approximation for the 
l-model, we analyse the tradeoff problem only for a restricted class of stationary admissible policies, 
where the definition of this class of admissible policies is motivated by the definition in Section 
4.1.2[ We assume that A < Smax as before. 


Stability : A policy 7 G F* is said to be stable if : a) the Markov chain Q[m] under 7 is positive 
Harris recurrent with stationary distribution tt^ on the recurrence class corresponding to go. and b) 

Q( 7 ,go) < 00 . 


Admissibility : In the following we restrict ourselves to the class of admissible policies F^ which is 
defined below. A policy 7 is called admissible if: 


RGl : it is stable, 

RG2 : it induces an aperiodic, irreducible Harris Markov chain Q[m], and, 

RG3 : the average service rate at a queue length g, E5(g) is non-decreasing in g. 


We note that the properties RGl and RG2 are similar to the properties of admissible policies used 
in Berry and Gallager [7|. The additional property RG3 is motivated by the monotonicity property 
derived in Agarwal et al. [T] as well as the monotonic non-decreasing property of any stationary 


deterministic optimal policy derived for the integer valued case in Section 4.1.2 We note that as 
in Section 4.1.2| for 7 G F^, the average queue length and average service cost are independent of 
go and are therefore denoted by < 5 ( 7 ) and C{'y) respectively. We also note that < 5 ( 7 ) = and 

c(7) = e. c,j(5(g)). 


We have that the TRADEOFF problem, for the R-model, is to obtain Q*{cc) which is the optimal 
value of 


minimize ( 5 ( 7 ), such that C'( 7 ) < Cc 


From the convexity of cr{s) and Jensen's inequality, we obtain that for any 7 G F^, C{'y) > cr{X). 
Similar to the proof of Lemma |4.3.4 it can be shown that there exists a sequence of policies 7^ G Fq 
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such that C'(7e) | cr{X). Therefore, we obtain that inf,ygra C'Ct) = c_r(A). In the next section, we 
obtain an asymptotic characterization of Q*{cc) in the asymptotic regime 5R as Cc i cji{X). 


4.4.1 Asymptotic lower bound 


We note that this problem can be considered as a special case of the tradeoff problem considered 
by Berry and Gallager [7], with the fade state taking only a single value. The contribution in this 
section is a step towards an alternative explanation for the Berry-Gallager lower bound, but with 
the extra property RG3 for an admissible policy. We note that asymptotic bounds for any order- 
optimal policy can be obtained under the additional assumption RG3. In Chapter 5, we present 
a lower bound for multiple fade states. As in [7j and as for the integer valued service case, we 
obtain lower bounds on the minimum average queue length for any sequence of admissible policies 
jk such that C{'yk) — cr{X) = f4 | 0. The asymptotic lower bound is obtained via a lower bound 
on the stationary probability for a Markov chain which evolves on M+. The lower bound is similar 
to Lemma 4.3.7f TAIL-PROB-STATE-DEP-l but for a Markov chain with state space M+. 


Lemma 4.4.1. Let {Q[m]) be the queue length evolution process for an admissible policy 7. Let 
Ca and 6a be as in assumption RAl. Suppose there exists a such that 


Vg G [0, qd],E[Q[m -L 1] - Q[m]\Q[m] = g] > -d, 

where d is positive. Then for any gi, /c > 0, A > 0, J > 0, A -|- (5 < Jq, and 0 < gi -L feA < q^, we 
have 




Pr {Q>qd}-^j f (ES'(g) - X)dTT{q) 

® Jqd 


The proof is presented in AppendixWe note that the development of the above lower bound 
on Pr {Q > q} is an extension of the geometric lower bound on stationary probability for countable 
space DTMCs available in Bertsimas et al. [8J and [QJ, to the case of DTMCs on M_|_ with state 
dependent drift. 


Using the above result, we derive the following asymptotic lower bound. Similar to assumption C2 
in Chapter 3 and as in [7j, we assume that the second derivative of cji{s) is positive at s = A. 


Proposition 4.4.2. For any sequence of policies 7 ^ G T^ with C{'^k) — cr{X) = f4 | 0, Q{^k) = 



Proof. Consider a particular policy 7 in the sequence jk with Vk = V. Let qd = sup {q : ES{q) < A -F ey}, 
where ey will be chosen later. Suppose qd is finite. From the admissibility of 7, we have that 
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Vg G [0,Qrf], ES'(g) < A + ey. Using d = ey in Lemma 4.4.1 we have for a g = /cA < qd, k > 0, 


Pr {Q >q} > 
Or we have that 


Sen 


Sea + ey 


+ 1 - 


Sen 


Sea + ey 


k r 


Pr{Q >qd} -/ (E5(g) - A)d7r(g) 


ey 


'Qd 


Pr {Q < q} < 1 - 


Sen 


Sea + ey 


1 /‘°° 

IH-/ iES{q) - X)d7T{q) - Pr {Q > qd} 

Jqcl 


. (4.26) 


We note that for q G [ 5 ^, 00 ), 'ES{q) — A > ey. For brevity, we denote ^ Jg“(E 5 (( 7 ) — X)d7r{q) by 
Dt- Now we note that for the policy 7, E 7 rCii;(S(Q)) — cr(A) = V. Define l(s) as the tangent to the 
curve cji(s) at (A, cji(X)). Then we have that E 7 rE 5 |Q[cij( 5 (( 5 )) — /(S'(Q))] = V. Now as cr{s) is 
convex and l{s) is linear, using Jensen’s inequality we have that E,r[cij(E5((5)) — ^(ES'((5))] < V. 
As in [71 step (41)], cji{s) — l{s) = G{s — X) where G{x) is a strictly convex function with G(0) = 0, 
G'(0) = 0, and G"(0) > 0. Thus we have that E^G(E5((3) — A) < 14. Using the sequence of 
steps (45), (46), (47), and (48) of Berry and Gallager f?], we obtain that 



(ES'(g) - X)d7r{q) 


2 


V 

< 

ai 


where ai > 0 is such that G(x) > aix^ (see Proposition 3.A.1), for x G [—X,Smax — A]. We note 
that then Dt < —\[^- Choosing ey = 4 a/^ we obtain that Dt < 4. 

^ — ey \l n-I o V \/ n.-i ^ — 4 


We note that ( 5 ( 7 ) > |, where q = sup {q : Pr {Q < q} < ^'}. Using the upper bound (4.26) and 
non-negativity of Pr {Q > qd}, if qi = kiA where ki is the largest integer such that 


1 - 


Se„ 


Sea + ey 


fci' 


[1 + A] < -, 


then qi < q. Therefore, ki is such that 

1 + 2A ^ ( Sea 

2 + 2Dt \ 5 eQ, -F ey y 

Sea J 1 + 2 A 


Let k 2 be the largest integer such that 


oe„ 


k2 


< 


1 + 2A 


(4.27) 


Then /c 2 < ki. We note that even if qd is infinite, k 2 A is a lower bound to q, since A is 0 in that 
case. The rest of the proof holds irrespective of whether qd is finite or infinite. 
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Then, from (4.27) and using the upper bound ^ on Dt, if ks is the largest integer such that 

2 


oea 


4 

ka 


< 


1 + 


1 ’ 


then ks < k 2 - We obtain that k^ is at least 




Since ( 5 ( 7 ) > | > we have that ( 5 ( 7 ) > ^ (i) “ 

log = 0 we have that for the sequence 7 ^ as 14 i 0 , Q{'yk) = ^ ( 7 ^) ■ ^ 


We note that the problem considered here is a special case of the Berry-Gallager problem (with 
a single fade state) with admissible policies. The upper bounds for the average queue length and 
average service cost for the TOGA policy from Neely |43], can be used to obtain asymptotic upper 
bounds for this problem. However, we note that these bounds hold only for the problem ( |4.4[ ), since 
the sequence of TOGA policies is not admissible. Therefore, as in Lemma 4.3. 10| we present a 
sequence of admissible policies which achieve the above asymptotic growth rate up to a logarithmic 
factor. 

Lemma 4.4.3. Let a policy 7 be defined as follows. At a queue length q, 7 serves a batch size 
min(g, s{q)), where 

/ 

X — ev, for 0 < g < q^, 

5 ( 9 ) = '^ A + ey, for q^ < q < 2q^, 

A + e, for 2qv < q. 


where qv > 0 and A+e < Smax- We obtain a sequence of policies ^k by choosing ey and from the 
sequence evi and q^ defined as follows. Let ojk = y/Vk, where I 4 i 0. Let ey, = 


oUAri 


aid W, = 


. Then we have that 7 ^ is a sequence of admissible policies, such that 


C(7fc) - c(A) = 0{Vk) and Q{^k) = 0 ( 7 ^ log (^)). 


The proof of this lemma is given in Appendix |4.J| and is motivated by and borrows ideas from the 
proof of the asymptotic upper bound for TOGA policies in 


The TOGA policy is the same as that in Remark 4.3.12 except that for every m > 1, we have that 
the service batch size STocA['kn] is chosen as 


STOCA[kn\ = min argmin ■(/?c(s) — V7[m]s )•, Q[m — 1] ] . 

ySC[0,5'ma^ 


Then, from [431 Theorem 3 and Corollary 1], for the TOGA policy 7 as above, for P > Sn 
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w = 




, e = ^, and g = I; log (^), we have that 


g(7) = o(V;0iog(/3)), 
C(7) = c(A) + oQ). 


For a sequence of policies 7 ^, generated by choosing j3k = for a sequence I 4 i 0, we have that 
Q{lk) = ^ ( log ( k)) ^*( 7 ) = c(A) + O ( 14 ). The above bound is an upper bound for 


the optimal solution of (|4.4[). Furthermore, we note that the above upper bound is an upper bound 


to Q*{cc^k) for a sequence Cc,fc as in Remark 4.3.12 


4.4.2 R-model with a piecewise linear cost function 


We note that R-model with a strictly convex cr{s) is usually used as an approximation to l-model. 
Usually, the function cr{s) coincides with c(s) for s G {0,... , 5maa:}- But we find that there 
are differences in the asymptotic behaviour of Q*{cc) for l-model and R-model. We note that 
cr{X) < c(A),VA G (0,5max) and cr{X) < c(A) for A 0 {0,..., S’^ax — 1}. Furthermore, R- 
model suggests that Q*{cc) increases to infinity for all A G (0,5max) as Cc i cr{X). However, 
for Case 1, we see that g*(c(A)) is finite (note that cr(A) < c(A) in this case). For Case 3, 
with cr{X) = c(A) we have that Q*(cc) = ^ ) for the R-model, whereas for the I- 

V vcc-c(A) y 

model Q*{cc) = H ^ e -c(A) ) • R-model with a strictly convex cr{s) overestimates the behaviour 
of Q*{cc) for Case 1 and underestimates c(A) and Q*{cc) for Cases 2 and 3. In the following, 
we briefly outline a method to show that a better approximation for l-model, is R-model with a 
piecewise linear cr{s). The service cost function cr{s) is chosen as the lower convex envelope of 
the service cost function c{s),s G {0, ...,5max} for the l-model. With this choice of cr{s), the 
asymptotic behaviour of l-model and its approximation, R-model, is the same. 


We consider Case 2 first. We define si,Su, and the line l{s) as in Section 4.3.3 Consider any 


sequence of admissible policies 7^ with C{'yk) — cr{X) = I4 0. Then we have that Eci^(ES'(g)) — 
cr{X) < 14 . For a particular policy 7 in the sequence, 0 < e < s;, and = s/ — e we have that 
Pr {Q < Qs} < as in the proof of Lemma 4.3.8 We assume that Pr {4[1] < f} = 4 > 0 , 
for some 0 < A < A. We note that for the R-model, the queue evolution is on M+. We discretize 
M+ into a countable number of intervals ([0, ^), [y, A),...). Then the proof of Lemma 4.3.8 can 
be modified to show that Qijk) = XI ^log ^ complete illustration of this proof technique 


is given in Lemma 5.7.2[ 


For Case 3, we proceed as in the proof of Proposition 4.4.2 by defining to be sup {q : ES'(g) < A -F ey}, 
where ey is a function of P to be chosen in the following. We recall that Dt = ^ {KS{q) — A) d'K{q). 
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For the policy 7 we have that 


E[cr(E5(Q))-/(E5(Q))] 

(cK(E5(g))-/(E5(g)))d7r(g) 


1 

ey 



(ES’(g) - X)d7r{q) 


< V, 

< V, or, 

V 

< -, 

mev 


where m is the tangent of angle made by the line passing through (A — 1, cr{X — 1)) and (A, Ci?(A)) 
with l{s). Now we choose ey = ^ to obtain that Dt < Then we proceed as in the proof of 
to obtain that Qi'jk) = ^ (tt)' 

We note that, by construction R-model has the service cost function cr{X) = c(A). Furthermore, 
the asymptotic behaviour for R-model and l-model coincide for Cases 2 and 3. For Case 1, it can 
be shown that Qi'ju) is finite for R-model, so that Q*{cr{X)) is also finite. However, we do not 
have asymptotic lower bounds for this case. 


Proposition 


4.4.2 


4.5 Conclusions 

In this chapter, we have obtained an asymptotic characterization of the tradeoff curve Q*{cc) in the 
asymptotic regime 5R for a discrete time queueing model (l-model). This asymptotic characterization 
has been obtained using the insights obtained from the analysis of INTERVAL-^uCHOICE in Chapter 
3. We also consider a real valued approximation (R-model) to l-model, and compare the asymptotic 
results which are obtained for R-model with that for l-model. 

For l-model we observe that the cost function c(s) as a function of the average service rate s 
is piecewise linear. Then as for INTERVAL-/iCHOICE-2-2, we have three cases. For Case 2, 
motivated by INTERVAL-^CHOICE-2-2, we construct an upper bound to the stationary probability 
distribution for the queue length which is geometrically increasing, which leads to 0(log(y)) 
asymptotic growth for Q*{cc) as Cc i c(A). We note that this geometric upper bound on the 
stationary probability distribution can be obtained in general, even for Cases 1 and 3. However, 
for Case 3, motivated by INTERVAL-/xCHOICE-2-3, we expect that there is a set of queue lengths 
with high probability for which the average drift ES'(g) — A I 0 as f4 I 0. We then expect that the 
stationary probability of such queue lengths should be equal and 0{V). This intuition leads us to a 
refined bound on the stationary probability of the queue length, obtained by extending the bounds 
available in Bertsimas et al. 1|9J, from which we obtained the 0 (^) asymptotic growth for Q*{cc) 
as V 10. 

We note that Case 1 is similar to INTERVAL-/iCHOICE-2-l, however we are unable to obtain 
asymptotic lower bounds except for the restricted case where = 1 and for the set of non-idling 
admissible policies. A direct translation of the ideas from INTERVAL-/iCHOICE-2-l is not possible. 
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since for the discrete time model we cannot obtain a dominating policy 7 ' as for INTERVAL- 
//CHOICE-2-1. 


We note that R-model is similar to INTERVAL-/xCHOICE-l, and as in Case 3 above, motivated by 
the observation that for INTERVAL-/iCHOICE-l, the stationary probability of queue lengths occur¬ 
ring with high probability should be equal and O as F | 0 , we obtain a asymptotic 

lower bound on Q*{cc)- We comparing the asymptotic behaviour of Q*{cc) between l-model and 
R-model in their respective asymptotic regimes in Section [4.4.2 We observe that R-model with a 
strictly convex cji{s) overestimates the behaviour of Q*{cc) for Case 1 and underestimates c(A) and 
Q*{cc) for Cases 2 and 3. Therefore, we conclude that a more appropriate real valued approximation 
to l-model should have a cost function cr{s) chosen as the piecewise linear lower convex envelope 
of c(s). 

Since in our approach, we obtain bounds on the stationary probability of the queue length, we are 
able to obtain asymptotic bounds on any sequence of order-optimal policies in the asymptotic regime 
5R. These bounds provide intuition for the design of buffer-partitioning policies and are presented 
We also obtain that the minimum average queue length is Q (log (’^)) for Cases 


in Section 


4.3.5 


2 and 3, when the arrival process A[m] is ergodic. 

In (45] Section 4.8], it is observed that the drift plus penalty algorithm idles for certain values of 
the queue length. However, in Lemma 4.3.2 we have obtained that any optimal policy 7 ^ should 


be non-idling. Hence, the drift plus penalty algorithm has to be modified to be non-idling, for 
the models considered in this chapter. So Lemma 4.3.2| can be thought of as providing theoretical 
motivation for the place-holder method in (45] Section 4.8]. 


In the rest of the thesis, we use the above results to obtain asymptotic characterizations of some 
resource tradeoff problems arising in point-to-point communication links. 
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Appendices 


4.A Proof of Lemma 14.3.1 


Our approach is to verify the hypotheses of the single Theorem in Sennott |67J by showing that the 


Assumptions (1), (2), and (3*) of Sennott j67| are satisfied. The assumptions in Sennott [67] are 
as follows (note that the notation is as in |S7j): 

1. For every state i and discount factor a, the optimal expected total discount cost Va(i) is 


finite, 


A 


2. Let ha(i) 


Vaii) — Va(0). There exists a non-negative N such that —N < ha(i), for all 


states i and discount factors a, 

3. There exists non-negative Mi, such that ha{i) < Mi, for every state i and discount factor 
a. Let the transition probability under action a, from state i to j be Pij{a). Then for all i, 
T’jj(a(i))Mj < oo for an action a{i) feasible in state i. 

Assumption (3*) assumes that in addition, Ylj Pi,j{o)Mj < oo, for all a feasible in state i. Let 
C{i, a) be the single stage cost at state i, when action a is taken. If Assumptions (1), (2) and (3*) 
hold, the Theorem [67] states that: 

Theorem 4.A.I. There exists a constant g, which is independent of the state i, and a function 
h{i) with —N < h{i) < Mi, such that 



A policy / that attains the minimum in the RHS of the above equation is average cost optimal, 
with optimal average cost g. 

For proving that the Assumptions (1) and (3) are satisfied we use [671 Proposition 5(i)] which states 
that: 

Proposition 4.A.2. Assume that the Markov decision process has a stationary policy / inducing 
an irreducible, ergodic Markov chain satisfying /(*)) < °° (EZl Proposition 4, Condition 

(i)]), where vTj is the stationary probability for state i under /. Then Assumptions (1) and (3) hold. 

Consider the stationary deterministic policy 7 / which uses a batch size s{q) = min(g, Smax) when 
the queue length is q. Then, from assumption A2 we have that from any g > 0 the state 0 can be 
reached, since Pr {A[l] = 0} > 0. From state 0, any state g > 0 can be reached, which follows 
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from assumptions A1 and A2. Therefore 7/ is irreducible. We note that state 0 is aperiodic, 
therefore the Markov chain under 7^ is also aperiodic. 

We now verify the drift condition (10.13) in [37] by choosing V{q) = 2(5 ^ -a) which is non¬ 
negative if A < Smax- Let pq^qi = Pr {Q[m + 1] = q'\Q[m\ = q, S[m + 1] = s{q)}. We have that 

Q 02 I \2 _2 

- '"(«)) S -'?+ 2(X1-A) - "■ 

As the function c{q) = q \s near-monotone [37| and — ^A)“ < 00 we have from (371 Theorem 
10.3] that the Markov chain under the policy 7/ is c-regular, implying that it is also positive 
recurrent with invariant distribution tt. Then under the same policy we have that K.,^cis{q,s) < 00 
as Ci 3 {q,s) < q + l 3 c{Smax)- This implies that we have verified condition (i) of Proposition 4 of 
[67] and therefore from [671 Proposition 5] the first and third assumptions hold. We note that as 
A[l] < Amax, once Assumption (3) holds, Assumption (3*) is implied. 

To verify Assumption (2) of |67| it is sufficient to show that the optimal discounted cost Va{q) is 
non-decreasing in q for every discount factor a G (0,1). This proposition follows as a special case 
of (23 Lemma C.l] and therefore we claim that Va{q) is non-decreasing in q without proof. 

Then from (671 Theorem] there exists a stationary deterministic optimal policy 7^ with optimal 
average cost g*p satisfying the following ACOE: 

+ min ^ {c/3(g, s)-h EJ/3(g - s-P A[l])} , 

with the stationary optimal policy 7^ using a batch size s*p{q) at queue length q satisfying 

s}{q) = argmin {cp{q, s)+KJ^{q - s + A[l])} , 
s&{0,...,mm{q,SmaA} 

where Jj 3 {q) is the optimal relative value function. 


4.B 


Proof of Lemma 




Proof. The proof proceeds by contradiction. Let {Q[m\,m > 0) be the evolution of the queue 
process under any stationary deterministic optimal policy 7 ^ starting from initial state q^. We 
assume that 7 ^ is such that there exists a queue length qi > 0, qi e TZg, such that s*^{qi) = 0 . 
Then we present a perturbation to this policy, which leads to a history dependent policy 7 , which 
has a smaller average queue length as well as average service cost, which contradicts the assumed 
optimality of 7 ^. However, to compare Q{j) with Q( 7 ^) and C{j) with C'( 7 ^), we identify a 
delayed renewal process (X^) embedded in {Q[m]). For 7 ^ and 7 , we associate different reward 


158 






processes with (Xk) and use the Renewal Reward theorem to obtain Q( 7 ^), <^( 7 ^), Q{j), and 
C{7). 

Consider the evolution {Q[m]). The first cycle Xi of the renewal process {Xj^) is defined as follows 


Xi = min{m : Q[m] = qi}. 

At slot Xi, Q[m] enters the recurrence class TZ^* and does not leave TZ^* again. We note that Xi 
depends only on the initial state go as {Q[m]) is Markov. If Tg(gi) is the random time taken to hit 
gi starting from g, then Xi = Tqp(gi). Furthermore for 7^, Xi < 00. 

Now we note that by definition (5[Xi] = gi and 5[Xi + 1] = 0. Let T'^(l) be the smallest positive 
integer such that S[Xi + r^(l) + 1] > 0, i.e., there is service of at least one customer in the 
{Xi + r^(l) + 1 )*^ slot. As {Q[m]) is Markov, the distribution of T'^(l) given Q[Xi] = gi, 
is independent of {Q[m],m < Xi). Let 5i C be the set of states in which at least one 
customer is served. The distribution of T’^(l) is the same as that of the smallest random time 
, to hit 5 i, starting from gi. Furthermore, we note that the queue length random variable 
Q(l) = Q[Xi +T,^(1)] is distributed as the state of the Markov chain at the random time 
when 5i is hit, starting from gi. The second cycle X 2 is defined as 

X2=r^^(l)+TQ(i)(gi). 


Similarly, the cycle 

Xk = — 1) + TQ(fc_i)(gi), 

where Q{k — 1) = Q[Xk-i +T^(A; — 1 )]. We note that Tq^{k)^k >1 are all IID and have the same 
distribution as . The random variables Q{k),k > 1 are all IID and have the same distribution 
as . Hence, the random variables Xk,k > 1 are independent and Xk,k > 2 are identically 
distributed. Thus (Xk) constitutes a delayed renewal process. From property 05, we have that 

EXfc < 00. 


We now associate a queue cost and service cost process with the renewal process. In each cycle k, 
we define the queue cost as the cumulative expected queue length : 

Xk-l 

C® = ^ E [Q[Tk + m]\Q[Tk] = gi], 

m=0 

where Tk = Using property 05, we have that Vk > 2, < 00 with probability 1, as 

the optimal policy is c^-regular j37]. Furthermore in each cycle k, we define the service cost as the 
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Figure 4.2: Illustration of the queue evolution under policy 7^ and its perturbation 7. The first three renewal 
cycles Xi,X 2 , and Xs are also shown. 


cumulative expected service cost : 

[c{S[Tk + m])\Q[n] = qi] . 

m=l 

Again using property 05, we have that Vfe > 2, < 00 with probability 1 , as the optimal policy 

is c^-regular. 

We note that the both and are dependent only on Then from the renewal reward 
theorem we have that 

E^’ 

ECf 
E^' 




In the following, we perturb 7 ^ to obtain the policy 7 . 


cycle embedded in Q[m] are illustrated in Figure 4.2 


The perturbation, as well as the renewal 


We note that, given (5[1], S'[2],...), < 5 ( 7 ) and C'( 7 ) do not depend on the order of service for 
customers. Therefore, we can assume that at least one customer, chosen to be served at T^+i + 
Tq^{k) was present in the queue at T^+i. Also, from the definition of Tg^{k), the system is idle in 
the slots {Tfc_|_i,... ,Tfc_|_i +T^(A;)}. The perturbed policy 7 advances the service of one customer 
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served in the (T^ + T^^{k) + slot to the (T^. +1)*^ slot, for every cycle k. The policy does not 
change the service batch size used at any other slot. To implement 7 , the evolution of the queue 
under the unperturbed 7 ^ policy is simulated for the same arrival process, in order to ascertain 
the slots at which the service batch size is to be changed. The policy 7 serves one customer at 
the (Tfc+i + 1)*^ slot, and serves one customer less at the (T^+i +Tg^{k) + l)*^ slot. To obtain 
( 5 ( 7 ) and C{^), we use the same renewal process {Xk), but associate different per-cycle queue and 
service cost costs. In cycle k > 2, the new per-cycle queue cost is the cumulative expected queue 
cost 

Xk-l 

Ck =Qi+ +m]- l\Q[Tk] = qi] . 

m=l 

AsC® < using the renewal reward theorem we obtain that Q{j) < Q{'y*). 

For 7 , the cumulative expected service cost for a cycle k > 2, 

Xk 

Ck = c(l) + E E[ciS[Tk+m])\Q[Tk] = qi]+¥. [ciS[Tk + T^ik - 1) + l]|Q[rfc] = qi] . 

m=2,m^Tk+T^^ (fc-l)-l-l 

The difference is 

c( 0 ) + ciSln + - 1) + 1]) - c(l) - c{S[n + - 1) + 1] - !)• 

From convexity for any s > 1, c(s) — c(s — 1) > c(l) — c(0). Therefore, and, using 

the renewal reward theorem for 7 , we obtain that C{^) < ( 7 ( 7 ^). Therefore 7 ^ cannot be optimal. 

Flence, \/q G such that g > 0, s^q) >0- D 


4.C Proof of Lemma 14.3.4 


Proof. Case 1 ; We first consider the case when s; = 0 and s/ < A < We note that the 
stationary deterministic policy, s{q) = min(g, s^) achieves the minimum average service cost c(A). 
Furthermore, the above policy has finite average queue length. 

Case 2 : We now consider the case when s; > 1 and si < X < Su- Let a G (0,1) be such that 
asi + (1 — a)su = A -F e, where e > 0 and A -F e < s„. Let Sa,m be a sequence of IID random 
variables with distribution 

\ Si w.p. a, 

I Su W.p. 1 — a. 

Consider a stationary policy S{Q[m]) = min(Q[m], 5a,m)- We note that as c(.) is monotonically 
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non-decreasing, 


E^c{S{Q)) < ET,c{Sa,i) < c(A -I- e) = c(A) -|- me, 


where m = Qn squaring both sides of the evolution equation (|4.1D and taking expecta- 

Su Si I r 

tions with respect to the stationary distribution, we have that 

E^(5[m -I- 1]^ = [Q[?ti]^ -I- S{Q[m\)‘^ + A[m + 1]^ — 2S{Q[m\)Q[m\ — 2S{Q[m\)A[m -f 1] -f 2Q[m\A[m + 1]] 

which implies that 2E,r [Q[m]S{Q[m\) — Q[m\A[m -|- 1]] < E -h E [A[l]^] . 

Let Am = Sa,m — S{Q[m]), we note that Am < Su- Furthermore Am = 0 Q[m] > Su- Therefore 
Q[m]S{Q[m]) = Q[m]{Sa,m - Am) > Q[m]{Sa,m) - si- Hence we have that 


2 E^ [Q[m]S{Q[m]) - Q[m]A[m -F 1]] < ES^,^^ + EA[ 1 ]^, 
2E^ [Q[m]{Sa,m - A[m + 1])] - 2sl < ES^ i +EA[1]2, 
or as m —> oo, 2eE.jrQ < 2s^ -F E5^^i -F E 74 [l]^, 
E.Q< ^ [2sl + ESl,+EA[lf] . 


Since E^^Q is thus finite, we note that the above policy is admissible. By choosing e from a sequence 
of Cfc I 0 as fc t oo, we obtain a sequence of policies 7^^ G T^ such that C'(7ej,) — c(A) < mek and 
Qileu) = C* (^) ■ Therefore linifctoo = c(A). 


Case 3 : We next consider the case when 1 < s; = A = < Smax- Let a G (0,1) be such that 

aA -F (1 — a)(A -F 1) = A -F e, where 0 < e < 1. Again we let Sa,m be a sequence of IID random 
variables with distribution 


Sa,m ~ 


A w.p. a, 

A -F 1 w.p. 1 — a. 


(4.28) 


Consider a stationary policy 5(Q[m]) = mm{Q[m], Sa,m)- The rest of the proof for Case 3 is 
similar to that of Case 2, and we obtain that there exists a sequence of policies 7 ^^, G Tq such 
that limej, 4 .o C( 7 efe) = c(A). We note that for a sequence of | 0, we obtain a sequence of 
policies 7 ej, G Ta such that C'( 7 ej,) — c(A) < mek and Q( 7 ej,) = O For both cases 2 and 3, 

since C'( 7 ej,) — c(A) < mek, by redefining we can obtain a sequence of policies 7 ^^, such that 
-c(A) = Cfc and g( 7 ej = O □ 


4.D Proof of Lemma 14.3.5 

Proof. For a policy 7 G Tq, let qs = inf {q : ES{q) > si — e} for a fixed positive e < si. Then 
we note that from assumption G3, for q > qs, ES{q) > si — e. In the following, for q > qs, we 
find a lower bound on the probability that at least one customer is served, i.e., a lower bound on 
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Pr {S{q) > 0}, given q> Qs- We note that for q> qs- 


ES{q) 

^max 

sPr{S{q) = s} 

s=0 

SmaxPr{S{q) > 0} 
Pr{S{q) > 0} 


> si-e, 

> si- e, 

> si-e, or, 
si-e 


> 


Sr, 


(4.29) 


For the policy 7, we have that vr = vrP where P is the transition probability matrix of the Markov 
chain under policy 7, i.e. Pqi,g 2 = Pr {Q[m + 1] = q 2 \Q[m] = qi}. We note that for a g > qg, 


from (4.29), we have that there is a positive probability of reaching a state less than q, starting 


from q in one step, i.e. 


P{Q[m + 1] < q\Q[m] = q) > 


si-e 

Sm.n.x 


Pr {^[1] = 0} 


Let Pd = { s {^[1] = 0}- 9 — Qs, from vr = ttP, we have 


For q = qs, we have 


For q = qs + 1, we have 


q-l g-1 

q'=0 q'=0 


Pr{Q<qs} > TT{qs)pd- 


Pr{Q<qs}+7:{qs) > -K{qs + l)pd, 

or Pr {Q < qs} > 7T{qs + l)pd- 


(4.30) 


Proceeding similarly, we obtain that for q = qs + k,k > 0 


l + TT 

7r{q) < Pr{Q < gj ^- - — = Pr {Q < gj —, 

Pd Pd 


where p = 1 + ^ > 1. 


Recall that in Chapter 3, we obtained a lower bound | on the average queue length for a policy 7, 
where g was such that Pr {Q < q} < We use the same idea here. Let g = sup{g : Yl\'=o '^W) ^ 
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j}. Suppose Pr{Q < (1 + -Pj < Let qi be the largest integer such that 




Pr{Q < Qs} + Pr{Q < gj ^ 


q=qs 


Then qi < q. We note that (4.31) is equivalent to finding the largest qi such that 

1 pQi-qs+i _ i 


Pr{Q <gj 


1 + 


or qi < logp 
Hence we obtain that the qi is at least 

logp 

Since Q{'y) > f > we have that 

Q(7) > I 


l + Pd{p-l) 


Pd p - 1 
1 

2Pr{Q < 


1 

“ 2 ’ 


- 1 


1 


2Pr{Q < qs} 


- 1 . 



1 

1 

^Ogp 

2Pr{Q < qs} 



(4.31) 


□ 


4.E Proof of Lemma 14.3.7 

TAIL-PROB : 


Proof. The proof of TAIL-PROB follows from that of Lemma 2 and Theorem 3 (2) of Bertsimas 
et al. [8], and is presented here for completeness. Let us define Q[m\ = max{qi,Q[m]) and Q = 
max(gi, Q). Then we note that as qi is finite, < oo. Hence, Q[m + 1] — Q[m] Q[m] 

0. We split this sum into three parts leading to : 


= Q 


qi-l 


0 = ^ 7r{q)E Q[m -L 1] - Q[m] Q[m] = q 

q=0 

+ vr(Q'i)E Q[m + 1] - Q[m] Q[m] = qi 


+ X] +1] “ QM QiM = Q 

q=qi+l 


(4.32) 

(4.33) 

(4.34) 


Now, as in [SJ we note that for g G {0, • • • ,qi — 1}, Q\m] = qi and Q[m-|-1] > qi. Therefore (4.32) 
> 0. Also we note that for G {o'! + 1, • •wo have that Q[m] = Q[m] and Q[m + l] > Q[m + 1], 
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so that (4.33) is bounded below by 


^ Ti{q)¥.[Q[m + l]-Q[m]\Q[m]=q]>-d ^ 7r(g). 

5=91+1 5=51+1 


Now let us consider (4.33). We have that (4.33) = 


E 


Q[m + 1] Q[m] = qi - qi> Emax(A[m + 1] - S{qi), 0) > l.e^, 


where assumption A2 is used. Substituting these lower bounds in (4.32) and (4.34) we have that 


0 > 'KiqiYa-d ^ 7r(g), 

5=51 + 1 

= eaPr {Q > gi} - eaPr {Q > + 1} - dPr {Q > gi + 1} . 


Hence Pr {Q > gi + 1} > 


-Pr{Q > qi} 


P d 

By redefining Q[m\ = max(( 7 i + l,Q[m]) we obtain that 


Pr {Q>qi+2}> — ^Pr {Q > qi + 1} 

£a I 


Induction leads to the following bound, for A; > 1 : 


Pr {Q>qi+k}> 


^aP d 


Pr{Q > qi}. 


□ 


TAIL-PROB-STATE-DEP-1 : 


Proof. We note that in this case the lower bound on Pr {Q > qi P k} is obtained in terms of 
expected drift over the tail of the queue length. Let us again define Q[m] = max(gi, Q[m]) 
and Q = max{qi,Q). Then we note that as qi is finite, Ej^Q < oo. Hence, we have that 


Q[^ +1] - <3N = d 

to : 


= 0. We again split this sum into three parts leading 


51-1 


0 = ^ 7r(q)E Q[m + 1] - Q[m] Q[m] = q 

5=0 

+ 7r(gi)E Q[m + 1] - Q[m] Q[m] = qi 


P ^ 7r(g)E Q[ni P 1] - Q[m] Q[m] = q 
5=51+1 


(4.35) 

(4.36) 

(4.37) 


As in the case of (4.32) and (4.33) we lower bound (4.35) by zero and (4.36) by 7r{qi)ea. Let us 


165 





























consider the case qd > qi- Then (4.37) can be written as 


Qd OO 

7 r(g')E Q[rn + 1] - Q[m]\Q[m] = q + ^ 7r(g)E Q[m + 1] - Q[rn]\Q[m] = q 
q=qi+i <?=ijd+i 

qd OO 

> -d ^( 9 )+ 7r(g)E[Q[m +1] - Q[m]|Q[m] = g] , 

q=qi + l g=<?d + l 


using the definition of qi and Q[m]. Using the lower bounds above on (4.35), (4.36), and (4.37) 
we obtain that 


qd 


0 > ea'K{qi)-d ■K{q)+ Y 7r(g)E [Q[m + 1] - Q[rn\\Q[rn] = q] , 

q=gi+l g=gd+i 

= eaPr {Q > qi} - SaPr {Q > qi + 1} - dPr {Q > + 1} + dPr {Q > qd + 1} 


+ Y^ 7r{q)E [Q[m + 1 ] - Q[m]\Q[m] = q], or, 
<?=9d+l 

Pr{Q>qi + l} > , Pr{Q > qi} H- '^Pr{Q>qd + l} 


Ca d 


- OO 

—t Y 7r(g)E[Q[m + 1 ] - Q[m]|Q[m] = g] 


9=9<i+l 


(4.38) 

(4.39) 


Redefining Q[m] = max(gi + l,Q[m]), where gi + 1 < qd, we obtain that 

Pr{Q>qi + 2} > , Pr {Q > gi + 1} H- ^^Pr{Q>qd + l} 

“r rx €a ~r Ci 


+ 


1 


+ d 


Y 7r(g)E [g[m + 1 ] - Q[m]|Q[m] = ^] . 


<?=g<i+i 

Induction leads to the following bound for qi + k < qd, k > 1, 

k 


Pr {Q >qi+k} > 


Ga P d 


Pr{Q > qi} 


1 - 


+ - 


€.a-\-d 


OO 


p d 


Pr {QPqdP^P 


^ Y 7r(g)E[Q[m + l]-Q[m]|Q[m] =g] 


P d 

g='?d+i 

We note that the bound also holds trivially for k = 0. Simplifying we obtain that for qi,k > 0, 
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such that qi + k < q^, 


Pr {Q >qi + k} > 


P d 


Pr{Q > qi} 


+ 1 - 


P d 


Pr{Q > gd + 1} + 


^ OO 

^(^)IE[Q[m + l]-Q[m]|Q[m] = g] 


<?=g<i+i 


TAIL-PROB-STATE-DEP-2 : 


□ 


Proof. The derivation of this bound is very similar to that of TAIL-PROB-STATE-DEP-1. We 


follow the steps in the proof of TAIL-PROB-STATE-DEP-1 till (4.39) with the drift d replaced by 


di. In this case, for 0 < < qd, (4.39) is further simplified to : 


Pr{Q>qiPl} > — ^^Pr{Q >qi} ^ - ^^^Pr {Q > qd P 1} - ^^^Pr {Q > qd P 1} , 

Ca + Ml Ca + Ml P “1 

since Vq > -|- 1, E [Q[m -El] — Q[m]\Q[m\ = g] > —^ 2 . Hence, we have that 

Pr{Q>qi PI} > ^ Pr{Q> qi} - Pr {Q > gd + 1} • 

+ Ml Ca + Ml 

Again by induction as before, we obtain that for qi P k < qd, k > 1, 


Pr{Q>qiPk} > Pr{Q>qi}- (l - ^ ^ ^ 


Ca + di 


Ca + dl 


di 


Pr {Q>qdPl}- 


The above lower bound also holds trivially for k = 0. 


□ 


4.F An upper bound on ^( 7 ) for a policy 7 

Let 7 be such that at a queue length q, a batch size mm{q, S{q)) is served, where S{q) < Smax 
is a random function of the current queue length. Furthermore, let 7 be such that E 5 (q) is a 
monotonically non-decreasing function of g such that there exists a finite queue length qi such that 
E5(gi) > A -F e, where e > 0. Then we have the following upper bound on < 5 ( 7 ). 

Proposition 4 .F.I. For a policy 7 as above, we have that 

g(^) < + m-^fPSlax ^ 
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Proof. The proof is very similar to that of 2.A.1 Let L{q) = < 7 ^ be a Lyapunov function. Then the 
Lyapunov drift 


A{q) = E[L{Q[m + 1]) - L{Q[m])\Q[m] = q] , 
< -2qiES{q)-X)+EA[lf + Si,,. 

We note that for q > qi, since ES{q) > A + e, we have that 

A{q) < -2q{e) + Eyl[l]^ + S‘^ax- 


For g < gi, we have that 


A{q) < —2g(e) + 2g [A + e — E 5 (( 7 )] + EA[1]^ + 

< -2q{e)+2qi{\ +e)+EA[lf‘+ 


Hence, 'iq, we have that 


A{q) < — 25 (e) + 25 i(A + e) + EA.[1]^ + 5", 


2 

max' 


Now applying (36l Theorem A.4.3] we have that 




□ 


4.G Proof of Lemma 14.3.10 


Let L{q) = be a Lyapunov function. Since for the policy 7, the batch size S{q) could be 

more than 5, the queue evolution equation under 7 is written as 

Q[m + 1] = max(Q[m] — S{Q[m]), 0) + A[m + 1]. 

The expected Lyapunov drift is 

A{q) = E [L{Q[m + 1]) - L{Q[m])\Q[m] = q] . 
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Since {A[m]) is assumed to be IID, as in |431 Lemma 5(a)] we have that 


A(g) < r]Ee^(5(<?)-^[i]) _ i 


As in the proof of [421 Lemma 5(a)] we have that 


2^2 


^g.;(5(g)-A) < ^ ^ cu(E5(g) - A) + ^ 


Hence, we have that 


A(g) < [(E<5(g) - A) + AT 


where K = 


^-^max ^ixlAjxiax 


Now by definition, the policy 7 is such that 


S(g) = 


St, for 0 < g < qt,, 
Su, for < q. 


Then we have that for q < qv 


A{q) < - Si - K] 


And for q> q^, 

A{q) < [(s„ - A) + K], 

= [X-si-K]+ [su - si] , 

< [X - Si - K] + uj [s„ - si] , 

Hence, for all q we have that 

A{q) < [X- si- K]+uj [s„ - s/] . 


We choose ui such that K < X — si. Proceeding as in the proof of (421 Theorem 3(c)], we have 
that 


wMqx-Q) < 

- (X-si-K)- 
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Since > E [e‘^(9.-Q)|Q < si] Pr {Q < s;}, we therefore have that 

Pr{Q < (4.40) 

{X- si- K) 

Now we note that 

C(7) = E 7rs(s)c(s) + TTs{si)c{si) + 7rs(s«)c(s„), 

S<Sl 

< Pr{Q < sj c{si - 1) + c(A) (1 - TTs{si) - ITsiSu)) + m [s„7rs(Sn) + SlTTsisi) - XilTsiSu) + 7rs(Si))] , 
where m is the slope of l{s). Then, we have that 

C( 7 )-c(A) < [c(si - 1) + mA - c(A)] Pr {Q < sj . 


Now consider the sequence of policies 7 ^ for which = log j foi" 3 sequence 14 < 1 such that 
14 i 0. Then we have that C'( 7 ) — c(A) = 0(14). Furthermore, from Proposition 


4.F.1 


we have 


that < 5 ( 7 ) = O ^log th3t 7 fc is also a sequence of admissible policies, since s{q) 

is a non-decreasing function of q and Q{'yk < 00 . 


4.H Proof of Lemma 14.3.17 


Proof. Define Q[m\ = max(gu, (5[m]) and Q = max(gu,Q) for a > 0. Then for any finite qu 
we have that KQ < oo and therefore 


7r(g)E Q[m -F 1] - Q[m] Q[m] = q 

q=0 


= 0 . 


We note that this can be written as 


0 = ^ 7r(g')E Q[m -F 1] - Q[m] Q[m] = q 

Q<Q u -^max 

+ ^ 7r(g')E Q[m + 1] - Q[m\ Q[m] 

Qu-A 

max <q<q u +A 

max 

+ ^ 7r(Q')E Q[m + 1] - Q[m] Q[m] = q 

qu-\-A. 

max <q 


= q 


(4.41) 

(4.42) 

(4.43) 


Then we note that for (4.41) as q < qu — Amax, Q[m] = qu and Q[m -F 1] < qu, so that 
Q[m -F 1] = qu- Therefore (4.41) = 0. Now consider ( |4.43 ). We note that for q > qu + Amax, 
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both Q[m] and Q[m + 1] are equal to Q[m] and Q[m + 1] respectively. Therefore we have that 


( |4 43D = Tr{q) [A - s{q)] , 

max <q 

^ ('Stt X)Pt {Q > q^i + • 


We also note that for qu — A^ax < q < Qu + A^ax, Qlm + 1] — Q[m] < A[m + 1]. Therefore, 


( 4.42 ) < \Pr {qu - Aynax <Q <qu + -^max} 


Using these upper bounds in (4.41), (4.42), and (4.43) we obtain that 


0 < XPr {qu - A max < Q < qu P A 

max } - (s„ - X)Pr {Q > qu + A 

max} 1 


which can be written as 


0 ^ XPt {Q > Qu Auiax{ {Q ^ Qu P Auiax{ 5 or, 

A 


Pr {Q > Qu + A 

max } < —Pr {Q > Qu + A 

max} ■ 

Sni. 


( 4 . 44 ) 


A A 


Let pu = —. Using (4.44) and inducting we obtain that 


Pr{Q>qu} < pi 


r_2«_1 

I ‘^‘A.max I 


(4.45) 


for any > Ay 


By redefining qu to be qu P k, k > 0, we can show that 


Pr {Q > qu^i Pk} < pi 


r Qu + k 1 

I 2Am,aa: | 


□ 


4.1 Proof of Lemma 14.4.11 


Proof. This proof is similar to that of the proof of Lemma 4.3.7 TAIL-PROB-STATE-DEP-1. We 
define Q[m] = max(gi, (5[m]) and Q = mayi{qi,Q). We note that since the policy is admissible 
and qi is finite, < oo. Therefore 



E 


Q[m + 1] - Q[m] Q[m 


q 


d'K{q) = 0. 
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We again split the above integral into three parts which leads to 


rgi-A 


/O 


0=1 E Q[rn + 1] — Q[m] Q[rn] = q dTr{q) (4.46) 

(4.47) 

Q[m + 1] - Q[m] Q[m] = q dTr{q). (4.48) 

where A > 0 will be chosen in the following. We note that for q G [0,gi — A) we have that 




+ / E 
J gi-A 

roo 

+ / E 

Jqi 


Q[m + 1] - Q[m] Q[m] = q dTr{q) 


Q[m] = qi and Q[m + 1] > qi, so that (4.46) > 0. Now as qi < qd and Q[m] > Q[m] we obtain 


that (4.48) 


> -dPr{qi <Q<qd)+ E [Q[m + 1] - Q[m]\Q[m\ = q] d7r{q) 

Jqd 


To obtain a lower bound on (4.47) we note that for q < qi, Q[m] = qi and Q[m + 1] > gi. So 
Q[m + 1] — Q[m] > 0. Then as in [71 steps (34), (35), and (36)] we use Markov inequality to lower 


bound E 


Q[m + 1] - Q[m] Q[m] = q ,q £ [qi - A,qi). 


E 


Q[m + 1] - Q[m] Q[m] = q 


Q[m] = g| 


> 5Pr ^Q[m + 1] — Q[m] >5 

> 5Pr {Q[m + 1] — Q[m] > (5 + A\Q[m] = q} , 

= 6Pr {A[m + 1] — S[m + 1] > (5 + A\Q[m] = q} , 

> dPr {A[m + 1] - Smax > <5 + A\Q[m] = q} , 

> Sea- 


We note that A and <5 have to be chosen so that A + 5 < (5a- Thus we obtain that (4.47) > 


6eaPr {qi — A < Q < gi}. Combining these bounds and using E [Q[m + 1] — Q[m]\Q[m] = g] = 
E5(g) — A, we obtain that 

poo 

0 > ScaPr {qi - A < Q < qi} - dPr{qi < Q < qd) + / E [Q[m + 1] - (5[m]|(5[m] = g] (i7r(g), 

J 


Pr{Q > gi} > 


Se„ 


-Pr{Q > gi - A} + 


dPr {Q >qd}- / (ES(g) - X)dTr{q) 
dqd 


d€a + €V dCa + d 

Similarly, if we define Q[m] = max(gi + A, Q[m]) and if gi + A < g^, we obtain that 

poo 

dPr{Q >qd}- / (E5(g) - A)d7r(g) 

*7 Qrl 


S 1 

Pr {Q > gi + A} > -— ^ — Pr {Q > gi} + 




Sea + d 
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By induction, we obtain that if /c > 0, and qi + k/S. < qa, then Pr {Q > (?i + kA} 


Sea 

Sea + ev 


Pr{Q > gi} + 


Pr{Q>qi}+{l- 



Sea 


Sea + ev 


poo 

dPr{Q >qd}- / i^S{q) - X)d7r{q) 

J<ld 

V {Q>qd}-\ f (lES’(g) - X)dTT{q) 
^ Jq^ 


□ 


4.J Proof of Lemma 14.4.3 


Consider a particular policy 7 in the sequence of policies jk- We note that the policy 7 is a stationary 
deterministic policy. Then, we have that 


<^( 7 ) = / CR{s)d7rs{s). 

Jo 


Since cji{s) is a non-decreasing function, we have that 


C'(7) < Pr {S < X - ev} c(A - ey) -|- tTs{X - ey)c(A - ey) -|- tTs{X -|- ey)c(A -|- ey) -|- 7rs(A -|- e)c(A -|- e). 

dc{X) 


< Pr {S' < A — ey} c(A) -|- 7r5(A — ey) ^c(A) — ey —' -|- 0(ey) 

+7rs(A -|- ey) ^c(A) -|- ey ^ -|- 0(ey)^ -|- '7rs(A -|- e) ^c(A) -|- e ^ -|- Cr(e)^ , 

where G(e) = c(A -he) - (^c(A) -h ■ 

Since there exists a finite queue length at which a service rate greater than A is used, we have that 

rX-ev 

Jo 


sdiTg^s) -|- (A — ey) TTsiX — ev) -h (A -|- ey) vrs(A -|- ey) -|- (A -|- e) tTs{X -he) — A, or, 


-ey7rs(A - ey) -h ey7rs(A -h ey) -h e7rs(A -h e) = 


r*A—ey 


sdTTsis) < XPr {S < A — ey} 


Hence, we have that 


dc{X) 


C( 7 ) < c(A) + Pr (S < A - ey} A^^ + G(e)7r,(A + e) + O (e^) . 

We note that 7rs(A -h e) = Pr {Q > 2q,;} and Pr {S < A — ey} = Pr {Q < A — ey}. We proceed 
to find upper bounds on Pr (Q < X — ey} and Pr {Q > 2qa}. 


We first obtain an upper bound on Pr {Q < A — ey} as in the proof of Lemma 4.3.10 Using the 
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Lyapunov function L{q) = we have that the Lyapunov drift is 

A(q') < — A + iT), 

where K = ^ Jhen, for g < g„, we have that 

A(^) < (-ey + i^) • 


For Qv < q < “^Qv, we have that 

A{q) < (ev + i^), 

= (-ey + it:) + 2a;e"'('?”-'?)ey, 

< {—ey + K) + 2ioev- 


For 2qy < q, we have that 

A{q) < {e + K), 

= {-ey + K) + (e + ey), 

< {-ey + K) + 2uj{e + ey). 


Therefore, for every q, we have that 


A{q) < {-ey + K) + 2ui{e + ey) 

Let K = ey j2. Or we have that ey = . Then, 

A{q) < -a;e‘"(^“-'?)^ + 2w(e + ey). 


Now as in the proof of Lemma 4.3.10 we have that 


Pr{Q < A- 


A - ey} < ?ii±Me-^(9«-A+Ev)_ 
ey 


Let us choose qv = log j ■ Then we have that 

Pr{Q<\-ey} = 0 {el) = O {u^) . 


(4.49) 


To obtain an upper bound on Pr {Q > 2qy}, we proceed similarly but with a Lyapunov function 
L{q) = The expected Lyapunov drift is 


A{q) = E \e^(Q[^+^]-<iP _ = q 
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Then we have that 


A{q) = 


^uj({q-s(q)) + +A) _ ujq 


As in [431, we have that 


A{q) < 


^uj{q-s{q)+A) _ ^U}q 




where Sjjiax — -^max T ^max- Or we have that 


^-uq^+LoSr, 


A{q) < e-‘^isiq)-A) _ ^ 

< -cje^O-g^E [(s{q) - A) - K] + 

= [s(g) _x_K] + , 


where K = 


— ^U) Amax 


For q < qv, we have that 


< -we'^O-g.) - i^] + 2wey + 


For q > q^, we have that 


A(g) < -cue'^O-'?*') [g^ _ + e-‘^9.+‘^<5. 


So for all q, we have that 


-we'^O-gO [g^ - AT] + 2wey + 


Since A' = 6^/2, we have that 




E 


^uj{q-qA 


^uj{q-qA 


< 2u}€v + 

< 4+- 


ujev 


Since e'^^'^Pr {Q > 2q^} < E [e‘^0 'J”)], we have that 


Pr {Q > 2qy} < 4e + e 


2g-t<^gD+(^<5 max 

uev 
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Since qv = ^, log ( ^ ) ■ Then we have that 


Pr {Q >2qy} < 4ey + 


Therefore, from the above upper bound and ( |4.49 ), we have that both Pr {Q < X — ey} and 
Pr {Q > 2qy} are ©(e^). Since O (ey) = C)(io), we have that both Pr {Q < X — ey} and 
Pr {Q > 2qy} are O (o;^). 

For the sequence of policies 7 ^, we have that ( 7 ( 7 ^) — c(A) = O (a;|) = 0(14). Since = 

with 


^ (i^)) ’ ~ ^ ( 4 ^ (it)) ' Therefore, using Proposition 

qi = 2qy, we have that Q( 7 fe) = O (^^log 


4.F.1 


We note that the policy 7 ^ is admissible. 

1 ( 1 


Therefore, we have a sequence of admissible policies 7 ^ such that Q{'^k) = O log (^17 jj ■ 
C{jk) = 0{Vk). 
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CHAPTER 5 


On the tradeoff of average power and average delay 

for fading wireless links 


5.1 Introduction 


Minimizing the average power as well as the average delay is a major requirement in current wireless 
communication networks, which brings the problem of designing good scheduling and power control 
policies to the forefront. In this chapter, we consider the characterization of the optimal tradeoff 
between average power and average queue length for a fading point to point link, with and without 
admission control, and obtain bounds on the tradeoff of average power and average delay by applying 
Little’s law. The models that we consider capture some of the important issues underlying the 
general problem for wireless networks: there is bursty arrival of traffic which can be subjected 
to admission control, the channel gain varies unpredictably, and the transmitter can dynamically 
change its transmission rate by varying the transmission power. The bounds that we derive are 
obtained using the methods discussed in Chapter 4, and are asymptotic in nature. However, unlike 
the models in Chapter 4, where there was only a single environment state and no admission control, 
here we consider models with multiple environment states as well as with admission control. We 
also consider the asymptotic characterization of the tradeoff for models with multiple queues in this 


chapter. The glossary of notation that we use in this chapter is given in Table 5.1 
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Symbol 

Description 

m 

slot index 

R[m] 

random number of arrivals in slot m (before admission control) 

A[m] 

random number of arrivals in slot m (after admission control) 

A 

■^max 

maximum number of arrivals in any slot 

A, 

mean and variance of yl[l] 

n 

set of fade states 

TTh 

distribution of fade state 

H[m] 

fade state in slot m 

Q[m] 

queue length at the start of {m + 1)*^ slot 

S[m] 

batch service size in slot m 

Smax 

maximum batch service size 

cj[m] 

history of queue evolution 

7 

policy - (S’[l],5[2],---) 

r 

set of all policies 

Ts 

set of all stationary policies 

Qii) 

average queue length for a policy 7 

Pih,s) 

power expended as a function of fade state h and batch size s 


average power for a policy 7 

Pc 

average power constraint 

c{\) 

minimum average power required for queue stability for 1-model 

cr{X) 

minimum average power required for queue stability for R-model 

Ta 

set of all admissible policies 

s{q) 

average service rate at a queue length q 

Q*{Pc) 

minimum average queue length over Fa under constraint Pc 

(-a 

probability of A[m] exceeding Smax 

A{-f) 

average throughput for a policy 7 

TT 

stationary distribution for a policy which is clear from the context 


stationary distribution for policy 7 


Table 5.1: Notation used in this chapter. 
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5.1.1 Methodology 


As stated in Section 4.1.1[ the scheme for obtaining asymptotic lower bounds in this chapter is very 
similar to that in Chapter 4. We briefly summarize the differences. We again obtain q which is the 
largest q such that Pru{Q < q} < ^- However, we note that geometric bounds for Pru{Q < q} 
are obtained as functions of the average drift of the queue length, where the averaging is done over 
the slot fade state also. For the tradeoff problem in this chapter, again a non-negative function 
D{q) will be obtained, where E^D((5) is the difference between the average power (rather than the 
average service cost) and c(A). For illustrating the method, we again consider the case of integer 
valued queue evolution and obtain a geometric upper bound Pr^lQ < q}. The steps which were 
followed for obtaining the r2(log(^)) asymptotic lower bound for the example in Section 


4.1.1 


can then be directly applied to obtain an Q (log (^)) asymptotic lower bound for the tradeoff of 
average delay with average power. 


5.1.2 System model - Integer valued queue length evolution 


We consider a discrete time system with slots indexed by the positive integer m. We assume that 
there is no admission control, so that A[m] = In each slot m, a random number of 

packets A[m] G Z+, where each packet is of the same size, arrive into the transmitter queue. 
The arrival sequence {A[m],m > 1) is assumed to be IID with A[l] < Amax, batch arrival rate 
Ej4[1] = a < oo, var(A[l]) = cj^ < oo. The packets are assumed to arrive into an infinite buffer, in 
which they wait until they are transmitted over a point to point fading channel. The fade state is 
assumed to be constant in a slot. The fade state takes values in a finite set T-L, with min {"If} > 0, 
and the fade state process {H[m],m > 1), is assumed to be IID, with H[l] ~ tth- The expectation 
with respect to tth is denoted by E^^^. The processes {A[m]) and {H[m]) are assumed to be 
independent of each other. 

The number of customers in the queue at the start of the (m -F 1)*^ slot is denoted by Q[m]. The 
system is assumed to start with Q[0] = go £ '^+ customers. At the end of slot m, a batch with 
^[m] G Z_|_ packets is removed from the transmitter queue just before the A[m] new packets which 
arrive in the slot are admitted. We assume that 5[m] < min {Q[m — 1], Smax), where Smax is 
the maximum batch size that can be served. The queue evolution sampled at the slots is given by: 


Q[m -F 1] = Q[m] — S[m -F 1] -F A[m + 1]. 


(5.1) 


The evolution of the queue length is illustrated in Figure 5.1 


At the start of slot m, the history of the system is defined as: 


a[m] ^ (go, H[l], S[1],Q[1], H[2], S[2],Q[2],... ,Q[m - 2],H[m - 1]). 


179 







H[l] 

H[2] 

iT[3] 


iT[m + 1 ] 

'■K 




>■ 





; A[m + 1 ] ; 


Q[0] fQ[^] (V[2] Q[3] Q[m] ('Q[m + 1] 

S'[2]^ 5[m + 1] = S'(Q[m^, i/[m + 1]) 


Figure 5.1: Evolution of the queue length Q[m]', the batch size ^[m] is chosen as a randomized function of 
the fade state H[m], the queue length Q[m — 1], and the history of the process a[m]. 


At the beginning of slot m, the scheduler observes H[m] and chooses a batch service size S'[m] G Z+ 
as a randomized function of the history cj[m], the current queue length Q[m — 1], and the current 
fade state H[m]. 


We define a policy 7 to be the sequence of service batch sizes (S'[l], S'[2],...). The set of all 
policies is denoted by T. If 7 is such that S[m + 1] = H[m + 1]), then 7 is a stationary 

policy. The set of all stationary policies is denoted as T^. In this chapter, we restrict attention to 
Ts in light of the discussion in Section 4.2. 1[ which also holds for this model. We also note that 
since H[m] is assumed to be IID, the process {Q[m],m > 0) is a Markov chain, if 7 G Tg. 


The transmitter expends P{h, s) units of power when transmitting s bits, when the fade state is h. 
We note that P{h,s) is a function of the fading gain /i^, when the fade state is h. Motivated by 
many examples (see [7] and [77]) of P{h, s), we assume that Vh G P, P{h, s) satisfies the following 
properties: 


Cl : P{h, 0) = 0, and 

C2 : P{h, s) is non-decreasing and convex in s, for s G {0,..., Smax}- 


The average power for a policy 7 is 


-P(7,W) = limsup^E 
M^oo M 


1 


M 


Y,P{H[m],S[m]) 


m=l 


Q[0] = go 


The average queue length for a policy 7 is 


<3(7 Wo) = limsup^E 

M^oo 


1 


M-1 

_m=0 


Q[0] = go 


(5.2) 


(5.3) 


We consider the optimal tradeoff of P( 7 ,go) with Q('y,qo) for this model. The optimal tradeoff 
between P( 7 , go) and average delay can be obtained from this using Little's law and is discussed in 
the following. We note that l-model considered in Chapter 4 is a special case of this model, with a 
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single fade state. 


5.1.3 System model - Real valued queue length evolution 

We state only the differences from the model discussed in the previous section. We assume that 
for m > 1, A[m] G [Q,Amax], 5'[m] G [0,5maa;], and qo G M+. Hence, the queue length Q[m\ G 
M+jVm > 0. The function P{h,s) is assumed to satisfy the following properties: 


RCl : P{h,0) = 0, for every h ^T-L, 

RC2 : P{h,s) is an increasing, strictly convex function in s, for s G [0,5maa:], for every h gT-L. 


The average power and average queue length are as in (5.2) and (5.3) respectively. 


The tradeoff of average power and average queue length, for such models without admission control, 
has been studied by Berry and Gallager [7], Neely [43], Goyal et al. |2^, Bettesh and Shamai [10], 
Biyikoglu et al. [77] as well as many others. We note that this model is similar to that considered 
by Berry and Gallager |7j, except that in |7], Smax = oo- We recall that Berry and Gallager obtain 
that any sequence of policies 7 ^, for which P{^k) is at most 14 rnore than the above minimum 
power required for queue stability, has Qi'jk) = ( 4 ^)' ^ model is 

similar to R-model considered in Chapter 4, except that in Chapter 4 we considered the case with 
a single fade state. 


In the following, the model in the previous section, where the queue length evolution is assumed to 
be on the non-negative integers, is called the l-model, while the model described here, where the 
queue length evolution is assumed to be on the non-negative real numbers, is called the R-model. 
We note that as in Chapter 4, R-model with fading and P{h, s) being strictly convex is usually used 
as an approximation for l-model. When used as an approximation, P{h,s) for R-model coincides 
with P{h,s) for l-model for s G { 0, . . . , Smax } ; V/l. 


5.1.4 An example 

Throughout this chapter, to illustrate the results for l-model and R-model, we use the following 
example. The number of packets 4[1] which arrive in a slot is assumed to be distributed according 
to a Binomial(4maa;,p) distribution, with arrival rate Vakb/s. The rate of service r in kb/s is 
assumed to be 2001og]^o (1 + SNR), where SNR is the received signal to noise ratio. We assume 
that SNR = where h'^ is the fading gain, P is the transmit power, and L encompasses the 
loss due to attenuation as well as noise power. 

We assume that the slots are of duration 2ms. We also assume that each packet has a size of 
100-bits. Then the arrival rate of packets in a slot is A = ^. We assume that if = 1 and 


181 




P = IW then r = 50. Therefore, if P{h,r) is the transmit power as a function of the fade state 
and the rate, we have that P{h,r) = — l). We note that in one slot, the number 

of bits served is 2r. We assume that the transmitter, in each slot, can choose its transmission 
rate in the set {0, 50,100}/cft/s. To fit this example to our model, we express the queue length 
in units of 100 bits. Then in each slot, we have a Binomial arrival process of 100-bit packets and 
service of s 100-bit packets, where s G {0,1,2}. The transmit power as a function of h and s is 
P{h,s) = }]^q50s/ 200 _ g g {0,1,2}. We note that the average queue length, as defined, 

is in units of 100 bits. 


For R-model, we assume the same distribution for A[l]. The set of possible batch sizes is as¬ 
sumed to be G [0,2]. The transmit power as a function of h and s is assumed to be P{h,s) = 
^ (1050V200 _ but for s G [0,2]. 

The above example uses a similar model for the rate as a function of the SNR as the example in 


Section 2.1.3 Flowever, in this chapter, we consider a slot level model, which models the system 
on a faster time scale compared to the continuous time model in Chapter 2. Consideration of the 
faster time scale is necessary since we are interested in the average delay advantage that can be 
achieved by scheduling the packets in accordance with the channel variations in each slot. 


5.1.5 Overview 


As in the previous chapter, the objective in this chapter is to obtain an asymptotic characterization 
of the minimum average queue length as the average power is a small V more than the minimum 
average power required for stability. We formulate the tradeoff problem for l-model and R-model 


in Section pf^ for a set of admissible policies, whose definition is similar to that in Chapter 4. We 
also obtain the infimum of the average power over the set of admissible policies, which is also the 
minimum average power required for stability of the queue, and discuss its properties in the same 


section. The asymptotic analysis of the tradeoff problem is then carried out in Section 5.3 For 
l-model, we show that depending on the value of A, three cases arise, which are similar to those 
for the l-model in Chapter 4. For the first case, through numerical experiments, we show that the 
minimum average queue length does not increase to infinity for an admissible policy which achieves 
the infimum of the average power over the set of admissible policies. For the second and third 
cases, as in Chapter 4, we show that the minimum average queue length increases as 0 (log (y)) 
and 0 (y) as 1/ {, 0. For R-model, we show that the minimum average queue length is 
We note that this is the same as the Berry-Gal lager lower bound, but the set of admissible policies 
that we consider is a subset of the set of admissible policies considered in |[7]. For the example 


in Section 5.1.4 we provide some numerical results to illustrate the bounds in Section ^ As 
in Chapter 4, we also obtain an asymptotic log (y) lower bound on the minimum average queue 


length for l-model, when {A[m]) and {H[m]) are assumed to be ergodic in Section 5.6 
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We then consider queueing models, l-model-U and R-model-U, which are similar to l-model and 


R-model, but with admission control, in Section 5.7 The tradeoff problems for l-model-U and R- 


model-U, are formulated in Section 5.7.2 and its asymptotic analysis is carried out in Section 5.7.3 


We show that the minimum average queue length increases as 0 (log (y)) when the average service 
cost is V more than the minimum, when 1/ | 0 and with a lower bound constraint on the utility of 
average throughput. 


We consider a single hop network model, which is an extension of R-model, in Section 5.8 


5.2 Problem formulation for l-model and R-model 


Our objective is to characterize the minimum average queue length for a given constraint Pc on the 
average transmit power. The following formulation is for both the l-model and the R-model. The 
tradeoff problem is 

minimize Q ( 7 , go)) such that P( 7 , go) < Tl;. (5.4) 

7er 


We note that as in Section 4.2.1 it is possible to formulate a CMDP for the above problem. The 
state space of the CMDP is Z+ x T-L, the action space at each (g, h) G Z_|_ x 77 is the set of batch 
sizes, and the evolution of the process is as given in igii. 


Similar to the quantities c(A) and cji{X) defined for l-model and R-model in Chapter 4, it is possible 
to obtain quantities c(A) and cr{X) for l-model and R-model discussed above. The quantities c(A) 
and Cij(A) can be interpreted as the minimum average power required for mean rate stability of 
the queue for l-model and R-model respectively. Then it is possible to show that if Pc > c(A) for 
l-model (or Pc > Ci^(A) for R-model), then there exists an optimal stationary policy 7 * for the 
above problem with stationary distribution ir*. So in the following we consider the above problem 
for policies in T^. 


For l-model, as discussed in Section 4.2.1 from Ma et al. 1351, if Pc is such that there exists a 


Lagrange multiplier > 0 and the average cost optimal policy 'ypp^ for a MDP with single stage 
cost q + (5p^P{h,s) in state {q,h) has T’( 7 / 3 p^,go) = Pc, then 7 ^^^ is optimal for ( |5.4[ ). The set of 
all such Pc is denoted as (similar to in Chapter 4). It can then be shown that for all Pc G 
there exist optimal stationary deterministic policies for ( |5.4D . Furthermore, such optimal stationary 
deterministic policies are such that the service batch size s{q, h) is monotonically non-decreasing in 
g for every h. Then, as in Chapter 4, we consider the above tradeoff problem only for policies 7 in 
an admissible set Tq for all values of Pc, where the definition of admissible policies is motivated by 
the above monotonicity property of any stationary deterministic optimal policy for Pc G C>“. Since 
R-model is an approximation to the l-model, the definition of admissible policies for R-model is 
motivated by the monotonicity property of any stationary deterministic optimal policy for Pc G 
for l-model. 
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We now state the properties which are satisfied by admissible policies for both l-model and R- 
model. A policy 7 is stable, if (i) the Markov chain {Q[m],m > 0 ) under 7 is irreducible, aperiodic, 
and positive Harris recurrent with stationary distribution vr, and (ii) Q(7, (70) < 00. A policy 7 is 
admissible if: 


G1 : it is stable, and. 


G2 : the average service rate in state q, s{q) = -fT) is non-decreasing in q 


B 


For an admissible policy 7, we have that the performance measures Q{'j,qo) and P{'y,qo) are 
independent of the initial queue state qo and exist as limits. Therefore, in the following, these 
performance measures are denoted by Q{'y) and P{'y) respectively. 

So in the following we consider the problem TRADEOFF: 


minimize <5(7), such that P{'y) < Pc 
■yeTa 


( 5 . 5 ) 


The optimal value of TRADEOFF is denoted as Q*{Pc). We note that whenever Pc G C>“, since 


there exists an optimal admissible policy Q*{Pc) is the solution to ( 5 . 4 ). 


For an admissible policy 7, we note that since the arrival rate is constant, from Little's law the 
average delay for 7 is The tradeoff of average delay with average power can be obtained as 

A ■ 


As in Chapter 4 , it is possible to consider a larger class of policies Ta,M, which is obtained by mixing 
or time sharing of policies in Tq. Let Q\j{Pc) denote the optimal value of the above problem, but 
with the minimization carried out over the set Ta,M- We note that the asymptotic behaviour 
for Q*j^{Pc) can be directly obtained from Q*{Pc). Therefore, in the following we consider the 
asymptotic characterization of Q*{Pc) only. 


If Pc is such that the above problem is feasible, then by definition there exists a feasible admissible 
policy 7 such that <5(7) < Q*{Pc) + e. Such a policy is called e-optimal in the following. 

We note that for any admissible policy 7, Q{'y) = Ej^Q and Pi'f) = EttEj^^iqE^iq S{Q, H)). 

Since Q and H are independent, we also have that ^(7) = E^Ett^E^iq S{Q, H)). For any 

7 G Ta, we note that the average arrival rate A has to be equal to the average service rate, i.e., 
A = EttEji^iqE^iqW) = E7rs((5). Therefore, for 7 G Tq, P{j) is lower bounded by the 


^We note for a stationary deterministic policy s{q,h) is non-decreasing in q for every h. Therefore, ¥,^gs{q, H) 
is non-decreasing in q. Since, we are considering randomized policies, we assume that ETrH^s\q,HS{q, H) is non¬ 
decreasing in q. 
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optimal value of 


minimize E^Eh\qEs\q^hP{H, S{Q, H)), 

such that E^EHiQEs\Q^HSiQ,H) = X, (5.6) 


since the only constraint is on the average service rate. Now we note that H) = 

E^^EQ\HEs\Q^HSiQ,H). Then, we have that 


^q\h^s\q,hS{Q,H) =11 sAp^^q^H-d'Kio) = s.dps^g\H = s dps^g\H, 


we have that ET^gEQ\HEg\QjjS{Q,H) = where the conditional distribution of S given 

H depends upon the policy. A similar procedure can be carried out on Et^Ej^^qE^^q^j^P^H, S{Q, H)) 
which leads to Et^Ejj\qEs\q hP{H,S{Q,H)) = Et^j^Es\jjP{H, S). Then the optimal value of 
(5.6) is bounded below by the optimal value of 


minimize ET^^Eg^HPiP^ S), (5-7) 

such that E.n-j:^E 5 |j|^S = A, 


where we minimize over all possible conditional distributions for the batch size S given H, irre¬ 


spective of the policy. For the l-model, we denote the optimal value of (5.7) by c(A), while for 


the R-model we denote the optimal value of (5.7) by cr{X) (we note that c(A) and cr{X) are 
the minimum powers required for mean rate stability for the l-model and R-model respectively, see 
|43j). We note that for the R-model, the conditional distribution of the batch size has support on 
[ 0 , Smax], while for the l-model the conditional distribution has support on { 0 ,..., Smax}- Hence, 
cr{X) < c(A),VA G [0,5maa;]- We note that feasible solutions exist for the above problem only if 
^ Smax- 


We have that V 7 G Tq, P{'j) > c(A) for the l-model, and P{'y) > cr{X) for the R-model. From 
[43l Theorem 1], we have that if A < Smax, then for every Pc > c(A) (or every Pc > cji{X) for the 
R-model), there exists an admissible policy 7 p^, such that P{'ypJ < Pc and as Pc i c(A), 
grows without bound. Since, for an arrival rate of A, c(A) (or cp(A) for the R-model) can be 
approached arbitrarily closely by admissible policies, c(A) (or cp(A) for the R-model) is the infimum 
of the average power for admissible policies. 


For the l-model, since properties (Cl) and (C2) are assumed to hold, from [421 Section VII], we have 


that c(A) is a piecewise linear (a proof is given in Appendix 5.A), non-decreasing convex function, 
for A G [OjS’max], with c(0) = 0. Again from |42j, cp(A) is a non-decreasing, strictly convex 
function of A G [0, Smax], with cp(0) = 0. For the example discussed in Section 5.1.4 the function 
c(A) and cp(A) are illustrated in FigurelS^ for different P and vrp as well as for the R-model and 


the l-model. We observe the following: a) for cases (i) and (iii) in Figure 5.2 for the same mean 
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fading gain for A < 1.2, having a fading gain larger than the mean with some positive 

probability leads to a smaller c(A); b) for case (iv) we see that cji{X) for the R-model is strictly 
convex. The function cr{X) < c(A), and coincides with c(A) for A G {0.5,1,1.5}; c) for both (i) 
and (ii) the function c(A) is piecewise linear, but the A-s at which the slope changes is different 
and depends on tth- We consider another example of c(A) and cr{X) for the example in Section 


From these examples, we can conclude for the R-model a smaller minimum average power cr(A) is 
sufficient for stability, compared with c(A) for the l-model. 


5.1.4 with Ti = {0.1,1} and 7rH(0.1) = 0.6. In this case, cr{X) < c(A) for all A 0 {0.4,0.8,1.4}. 



0 0.2 0.4 0.6 0,8 1 1.2 1.4 1.6 1.8 

I 


Figure 5.2: The optimal value c(A) of problem ( |5.7| ) for l-model: i) with H G {0.5,1} with 7r//(0.5) = 0.5 
and S € {0,1, 2}, ii) with H G {0.5,1} with 7rj:/(0.5) = 0.7 and S G {0,1, 2}, and iii) with S G {0,1, 2} and 
fade state fixed at 0.7906 = •\/0.5 X 0.5^ + 0.5 X P with the same average fading gain as (i). For R-model: 
(iv) with H G {0.5,1} with 7rij(0.5) = 0.5 and S G [0,2]. 


As shown in Figures 5.2 and 5.3 it is possible that CR(Aj) = c(Aj) for some Xi G [0, Smax]- We will 
see from the asymptotic analysis in the next section, that the asymptotic growth rate of minimum 
average queue length suggested by the R-model and the l-model for such Xi are different. We note 
that, in general, it is not known for what A*, if any, cji{Xi) = c{Xi). However, if iTf] = 1, it is clear 
that Cr{s) = C(s),\/S G {0, . . .,Smax}- 


5.3 Asymptotic bounds for l-model 

In this section, we obtain an asymptotic characterization of Q*{Pc) for l-model and R-model as 
Pc i c(A) and Pc {, cr(A) respectively. For l-model, as in Chapter 4, we assume 

A1 : Pr{A[l] > Smax} > Ca > 0. 

A2 : Pr{A[l] = a} > 0, for all a G {0,... ,Amax}, 
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Figure 5.3: The optimal value of problem ( |5.7[ ): c(A) for l-model and Cij(A) for R-model, with H G {0.1,1} 
and 7r//(0.1) = 0.6; c(1.7) is 107% of Ci^(1.7). 


For R-model as in Chapter 4, we assume 

RAl : Pr{A[l] - Smax > > ea, for positive 5a and ea- 

We note that part of this analysis was presented in |6IJ- We consider the l-model first. Consider any 
7 G Ta. To obtain the asymptotic behaviour of Q*{Pc) as Pc i c(A), we ascertain the asymptotic 
behaviour of 7r(g) as Pc i c(A). We note that 7r{q) determines the average queue length and the 
average power given the policy. As we have seen in Chapter 4, it turns out that the asymptotic 
behaviour of 7r(q), is determined by the average drift of the queue, X — s{q), at a queue length q. So 
we proceed by relating the average power to the average drift A — s{q). We note that the average 
power used when the queue length is q is S{Q, H)), which is bounded below by 

the optimal value of 


minimize 
such that 


E^^Es\hP{H,S), 
IEtthIEsIH'S' = s{q), 


where we have considered all possible conditional distributions on the batch size with support on 
(0,..., subject only to the constraint that the average service rate is s{q). The above 


optimization problem is the same as (5.7) except that the constraint is now s{q) instead of A. 
Therefore, the average power used when the queue length is q is bounded below by c{s{q)). We 
note that any feasible policy 7 for TRADEOFF has ^(7) < Pc- Then for that 7, E^c(s(( 5 )) < 
Pil) < Pc- We also note that from the convexity of c(s), E7rc(s((5)) > c(A), since E 7 rs(Q) = A. 
Now as Pc i c(A), for any sequence of feasible policies for TRADEOFF, Ec(s((5)) {, c(A). 
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Figure 5.4: Illustration of the relationship between A, si, and Su along with the minimum average cost c(A) 
and the line l{s) for the three cases 


The behaviour of Q*{Pc) as Pc i c(A) is observed to depend on the relationship of A with 
c{s),s G [OjiSmax]- Since c(s) is piecewise linear, we can define a sequence of intervals [ap,bp], 
p G P}, with Op+i = bp, ai = 0, and bp = Smax- The sequence of intervals is such that 

for s G [ap,bp], c(s) is linear. The following three cases arise: 


1. ai = 0 < A < 6i, 

2. Up < X < bp, p > 1, and, 

3. X = ap,p > 1. 


We note that cp{ap) = c{ap) in Figures 5.2 and 5.3 For Case 1, through numerical examples, we 
illustrate that Q*{Pc) does not grow to infinity as Pc c(A). We obtain an asymptotic lower bound 
for Case 2 which is the asymptotic lower bound to the super-fast log(y) upper bound observed 
for the sequence of policies constructed by Neely in 231 Corollary 2], but for admissible policies. 
We also obtain an asymptotic lower bound for Case 3 which is used to illustrate another difference 
between the asymptotic behaviours of Q*{Pc) in the R-model and l-model. 


For Cases 1 and 2, let si = Up and Su = bp, while for Case 3 let si = Su = a.^ 
shown in Figure 5.4 We define the line l{s) : [OjS'max] —t M+ as follows: 


An example is 


1 . 

2 . 


If s/ < A < Su, then l{s) is the line through {si,c{si)) and (su,c(s„)). 

If s; = A = Su = Op for some p> 1, then l{s) is a line through (A, c(A)) with slope m chosen 

such that <m< <^p+'^"i-<^p'^ 

dp dp —1 ff'p 


We note that E/(s(Q)) = c(A). We now present asymptotic lower bounds for Q*{Pc), which follow 


directly from the asymptotic lower bounds in Lemmas 4.3.8 and 4.3.13 


Lemma 5.3.1. For Case 2, given any sequence of admissible policies 7 a: with P{'yk) — c{X) = 14 0, 

we have that Qi'jk) = (log (t^))- 

^We note that unlike in Chapter 4, s; and Su need not be integers. 
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Proof. We first note that for the sequence 7 ^, since P{^k) — c(A) = f4, we have that = 


E 7 rc(s(Q)) — c(A) 4 , 0, with Uk < f4. The rest of the proof is very similar to that of Lemma 4.3.8 


With s{q) replacing ^S{q) and replacing f4 (with an inequality) in the proof of Lemma 4.3.8[ 
and m being redefined as the tangent of the angle made by the line passing through (ap_i, c(ap_i)) 
and (ap,c(ap)) with l(s), and proceeding as in the proof of Lemma 

^ 0°^ (a)) ^ ^ 0°^ ( 4 ))' 


4.3.8 


we have that Q(jk) = 

□ 


We note that an asymptotic upper bound for Case 2 can be obtained from a sequence of TOCA 
policies from |43|. However, TOCA policies are not admissible. Therefore, we now present an 
asymptotic upper bound for a sequence of policies, which is similar to the sequence of buffer 
partitioning policies proposed by Berry and Gallager [7]. 

Lemma 5.3.2. Let a policy 7 be defined as follows. When the queue length is q and fade state is 
h, 7 serves a batch size min(q, 5 (g,/i)), where 


S{q,h)^ 


pi{h), for 0 < g < q^, 
Puih), for qy < q. 


where qy > 0 , pi{h) and pu{h),\/h G P are the optimizing distributions for ( |5.7D with the constraint 

I, where 


being s; and Sy respectively. We obtain a sequence of policies 7 ^, by choosing qy = log 
Vfc < 1 is a sequence decreasing to zero. Then for Case 2, 7 ^ is a sequence of admissible policies, 
such that P( 7 fc) - c(A) = 0(14) and ^( 7 ^) = O (^log (^))- 


The proof of this asymptotic upper bound is given in Appendix |5.B We note that the proof is quite 
similar to that of Lemma 14.3.101 

We then have the following result. 

Proposition 5.3.3. For Case 2, we have that Q*{Pc,k) = © (^log (^ p ^ic(A) )) Pc,k i c(A) 

and Pc^k = P{lk) for the sequence of policies in Lemma 


5.3.2 


The proof is very similar to that of Proposition 4.3.11 and is therefore omitted. 


Remark 5.3.4. We now consider a sequence of policies generated from a modified form of the 
tradeoff optimal control algorithm (TOCA) JS!- We note that in |43J, the control variable is the 
power allocation, and the service rate is obtained as a function of the power allocation and the 
current fade state. The set of all possible power allocations (H in [43j) is the same for all fade 
states. However, in our model the control variable is the batch size. Therefore, we modify TOCA 
such that the set of power allocations is a function of the fade state. For each fade state, the 
possible power allocations are such that the batch sizes takes all possible values in { 0 ,... ,Smax}- 
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The modified TOCA algorithm is again parametrized by positive numbers w,e,q, and /3 as in 
The algorithm chooses at each slot m > 1, the batch size stoca such that 


STOCA['m] = min argmin ^ l3P{H[m], s)— W[m]s\ ,Q[m — 1] 

\ SG{0,...,5'Tnax} 


where 


W[m] =I{Q[m — 1] > q} ^^+2X[m—l] + I{(5[to — 1] < g} — ^^'>+2X[m — l] 


We note that stoca[t^] = 0 if fT[m] < 0. The sequence X[m],m > 0 is obtained from a virtual 
queue which evolves according to 


X[m + 1] = max(X[m] — STOCA[m + 1] — eI{Q[m] < g} , 0) + A[m + 1] + eI{Q[m] > q] . 


As in [431, let ^max = max(Amax, •S'^aa:)- For our Case 2, let 0 < e < min(A - ap,bp - A), 
w = , and q = f-\og{(3). A sequence of TOCA policies 7 ^ is generated by choosing 

^max ^ 

a sequence /3fc = ^, for a sequence 14 i 0. The proof of [431 Corollary 2] extends to the above 
version of the TOCA algorithm, and we have that 

Q(7fc) = O (log (^)) ,nik) = c(A) + O iVk). 


Therefore, we obtain that for the sequence of policies 7 ^, Q( 7 fc) = O ^log ■ We note 

that W[m] is a non-decreasing function of q, where Q[m — 1] = q. Since sroCAl'm] is a non¬ 
decreasing function of fT[m], we have that STOCA[fn] is a non-decreasing function of Q[m — 1]. 
However, 7 ^ is not a sequence of admissible policies, since the policies 7 ^ depend on the auxiliary 
variable X[m — 1]. We note that the above upper bound is an upper bound on the optimal value of 
Furthermore, as in Remark pl-.3.12[ for any subsequence of such that there exists a 
constant m > 1 and a subsequence PTOCA,k of P{'jk) such that PTOCA,k — c{X) < m {Pc,k — c(A)), 


we have that Q*{Pc,k) < Q{lk) = O (^log 


PTOCA,k—c{\) 




Lemma 5.3.5. For Case 3, given any sequence of admissible policies 7 ^ with P( 7 fc) — c(A) = I 4 4 , 0, 
we have that Q( 7 fc) = FI ( ^ ). 


Proof. We again note that for the sequence 7 ^, since P{'yk) — c(A) = I 4 , we have that Uk = 


E7rc(s((5)) — c(A) i 0, with Uk < 14 . The rest of the proof is very similar to that of Lemma 4.3.13 


With s{q) replacing E5(g) and Uk replacing I 4 (with an inequality) in the proof of Lemma [4.3.13[ 
and m being redefined as the tangent of the angle made by the line passing through {ap,c{ap)) 
and (ttp+i, c(ap+i)) with l{s), and proceeding as in the proof of Lemma 4.3.13 we have that 
Q{-fk) = n(j}^ since Vk<Uk. □ 
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Remark 5.3.6. The derivation of the relationship between the difference of the P{'y) and c(A) and 
the average drift defined as [Q["i + 1] “ = q] in the proof of Lemma 


by the choice of ey and then ey can be chosen so as to obtain the tightest asymptotic lower bound. 
However, in [7], qd cannot be chosen arbitrarily. In fact, qd is chosen as the queue length which 
has the maximal stationary probability of all queue lengths in the set {O,..., [ 2 ( 5 ( 7 )] } ^ policy. 

The freedom in the choice of qd enables us to derive the L2 (^) asymptotic lower bound. 

Remark 5.3.7. As in the analysis of Case 3 in Chapter 4, it is possible to show that a sequence 
of randomized policies { 7 fc} achieves the asymptotic lower bound derived above for Case 3. The 
sequence of policies is parametrized by a sequence f4 i 0. For a particular f4, the policy chooses 


4.3.13 is motivated by the approach in [7]. We note that in our proof, qd can be chosen arbitrarily 


S[m] = mines'{H[m]), Q[m — 1]), 


where S'{H[m]) is independently chosen for each m and is distributed according to any condi¬ 


tional distribution of batch size given H[m] which is optimal for (5.7) but with the rate con¬ 


straint being A -|- 14. For a particular I 4 , it can then be shown that P{'yk) = c(A -|- I 4 ) 
and Qi'fk) < ^ Then as A: — )■ 00 and I 4 i 0, Qi'jk) = O (^\t) 

P( 7 fc) = c(A) + 0(14). 


We have the following result. 

Proposition 5.3.8. For Case 3, we have that Q*{Pc,k) = © (^ p ^,ic(A) ) where i c(A) and 
Pc,k = P{lk) foi" the sequence of policies in the above remark. 


The proof is very similar to that of Proposition |4.3.11 and is therefore omitted. 


5.4 Asymptotic bounds for R-model 

We now consider the asymptotic behaviour of Q*{Pc) in the asymptotic regime JR as Pc i cr{\) for 
the R-model. Similar to the l-model, it can be shown that the average power used when the queue 
length is q is bounded below by cpis^q)). We again note that any feasible policy 7 for TRADEOFF 
has P(7) < Pc, and for that 7, E.n-Cij(s(( 5 )) < P(7) < Pc- We also note that from the convexity of 
cp{s), Et^cr{s{Q)) > cr{X), since E7rs((5) = A. Now as Pc | cr{X), for any sequence of feasible 
policies for TRADEOFF, Eci{(s((5)) i cp(X). We note that the following result is similar to the 
Berry-Gallager lower bound, but is derived with the extra assumption G2. 

Lemma 5.4.1. For any sequence of admissible policies 7 ^ with P( 7 fc) — cp(X) = 14 | 0, we have 
that Q(jk) = ^ (^) ■ Therefore, Q*{Pc) = ^ ^ ^p^!_c(a) ) cr(A). 
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Proof. We note that for the sequence 7 ^, since Pijk) — cr{^) = we have that Uk = 
IE 7 rCi{(s(( 5 )) — cr{X) 4 , 0, with Uk < Vk- The rest of the proof is similar to that of Proposi¬ 
tion 4.4.2| We follow all the steps in the proof of Proposition 4.4.2 with s{q) replacing ^S{q) 
and Uk replacing Vk (with an inequality) in the proof of Proposition 4.4.2 to obtain that Q{'yk) = 


ri ^ 'f'k ^ sequence of e-optimal policies for TRADEOFF for the sequence 


VvTt; 


Pc^k- Then we have that T’( 7 (,) i cr{X) and Q( 7 (.) = O , ^ = . Since 7 ^, is e-optimal, we 

\yPc,k — CRU)J 


have that Q*{Pc,k) > Qil'k) - £■ Therefore, Q*{Pc,k) = 5^ ( ,k-c{x) ) ^ cr{X). 


□ 


Remark 5.4.2. A sequence of admissible policies 7 ^ can be obtained which achieves the above 


asymptotic lower bound up to a logarithmic factor, as in Lemma 4.4.3 The proof of this upper 
bound is very similar to that of Lemma |5.3.2 and is therefore omitted. A particular policy 7 in 
the sequence is defined as follows. The policy 7 serves a batch size m.in{q, S{q, h) when the queue 
length is q and fade state is h, where 


p-{h), for 0 <q < q^, 

S{q, h) ^ { P+{h), for qv < q < 

Pe{h), for < q, 

where q^ > 0 , p-{h), p+{h), and p^{h),\/h G P are the optimizing distributions for ( |5.7D with the 
constraint being A — ey, X + ey, and A -F e respectively. We obtain a sequence of policies 7k, by 
choosing ey and qv from sequences ey^ and qv^. defined as follows. Let Uk = VVk, where Vk i 0. 


Let ey. = ujkAl^ 


o^An 


a'id qv^ = ^ log 


Remark 5.4.3. As in Remark 5.3.4[ for a sequence of policies 7 ^ generated by the modified form of 
TOGA, it is possible to show as in f43j, that for I 4 < Pk = ^, ^k = Wk = 
and% = 4log(j^), 


VPk ' 


P( 7 ) = c{X) + 0{Vk), 


as Vk i 0. We note that the above asymptotic upper bound applies only to problem (5.4), since 
the sequence of policies considered above is not admissible. 


5.5 A numerical example 


To illustrate the results obtained in the previous section, we plot the optimal tradeoff curve for the 


example in Section 5.1.4 We note that the results in this section apply only to l-model. We assume 
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that Amax = 5. We consider first a case where the fading gain T-L = {0.5,1} with 7r//(0.5) = 0.5. 


The tradeoff curves Q*{Pc) for A G {0.9,1.0,1.1}, are shown in Figure 5.5 We note that these 
arrival rates correspond to bit arrival rates of 45,50, and 55 kb/s respectively. Each point in the 
tradeoff curves is obtained by numerical solution of a suitably truncated MDP with state being the 
current queue length and the current fade state, and single stage cost q + f3P{h,s), where /3>0 


is a Lagrange multiplier. Each tradeoff curve is obtained by varying /3. From Figure 5.2 we have 
that c(0.9) = 1.20, c(l.O) = 1.384, and c(l.l) = 1.781. From the asymptotic characterization 
of Q*{Pc), we have that for A = 1.0, Q*{Pc) increases as l/{Pc — 1.384), while for A = 0.9 
and 1.1, Q*{Pc) increases as log 


1 


Pc-c(A) 


In Figure 


5.5 


we observe that for the same average 


queue length, the difference between the average service cost and c(A) increases from A = 0.9 to 
A = 1.0 and then decreases. This difference is even more pronounced in Figure [5^ where A is 


increased from 0.78 to 0.80 and then to 0.82 for the example in Section 5.1.4 with 77 = {0.1,1} 
and 7rH(0.1) = 0.6. 



Figure 5.5: The optimal tradeoff Q*{Pc) for the system in Section 5.1.4 with 77 = {0.5,1} and 7ri/(0.5) = 0.5, 


for A e {0.9,1.0,1.1} with c(0.9) = 1.2, c(l.O) = 1.384, and c(l.l) = 1.781. 



Figure 5.6: The optimal tradeoff Q* 


for A G {0.78,0.80,0.82} with c(0.78) = 1.0717, c(0.80) = 1.1071, and c(0.82) = 3.0995. 
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For the example in Section |5.1.4| with % = {0.1,1} and 7riif(0.1) = 0.6, for A = 0.82, we plot 
the optimal tradeoff curve Q*{Pc), the asymptotic lower bound from Lemma 5.3.1 (using Lemma 


4.3.8), and the upper bound for the sequence of policies from Remark 5.3.4 in Figure 5.7 We 


note that the upper bound is obtained via simulation, for a sequence of policies for which q has 
been chosen to be ^ of what is suggested in (El Theorem 3]. This heuristic has been used in 
[El- The analytical upper bound for the sequence of TOGA policies from [431 Theorem 3] is found 
to be very weak. We note that the asymptotic bounds, although tight in the order sense, are very 


weak. To illustrate the bounds for Case 3, for the system in Section 5.1.4 with H = (0.1,1} and 
7rH(0.1) = 0.6, for A = 0.8, we plot the optimal tradeoff curve Q*{Pc), the asymptotic lower bound 
^ from Lemma 5.3.5 (using Lemma 4.3.13), and the upper bound for the sequence of policies from 


Remark 5.3.7 in Figure [5^ Again the upper bound is obtained via simulation. 



Figure 5.7: Comparison of the optimal tradeoff Q*{Pc) with the lower bound from Lemma 
bound from Remark 


4.3.8 


and the upper 


5.3.4 


for the system in Section 


5.1.4 


for A = 0.82, for U = (0.1,1} and 7r//(0.1) = 0.6 



4.3.13 


and 


Figure 5.8: Comparison of the optimal tradeoff Q*{Pc) with the lower bound from Lemma 
the upper bound from Remark 5.3.7 for the system in Section 5.1.4 for A = 0.8, for H — {0.1,1} and 
7r//(0.1) = 0.6 


The numerical results illustrate that the asymptotic bounds which are obtained using the methods 
in Chapter 4 are weak, although they are tight in the order sense for Cases 2 and 3. 
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We consider the case A = 0.2 in Figure [5^ which corresponds to Case 1 for the example in Section 


5.1.4 for both % = {0.5,1} and % = {0.1,1}. We note that c( 0 . 2 ) is 0.1992 for both % = {0.5,1} 


and T-L = {0.1,1}. We observe that Q*{Pc) approaches a finite value in both cases. We note that, 
for Case 1, if \T-L\ = 1, it is possible to show that there exists a policy 7 such that P{^) = c(A) and 
( 5 ( 7 ) < 00 . 



Figure 5.9: The optimal tradeoff Q*{Pc) for the system in Section 
Q*{Pc) approaches a finite value in both cases. 


5.1.4 


for A = 0.2, for two cases of H', 


The exact nature of Q*(Pc) for the different cases depends on the shape of TT{q) in the regime 3?. 
We first consider Cases 2 and 3 for l-model. Intuition for the behaviour of Q*{Pc) for R-model is 
similar to that of Case 3 for l-model. In Figures 5.10| and 5.11 we illustrate the behaviour of the 
probability mass function (PMF) of the queue length for Cases 2 and 3. For each case, the PMF 
has been obtained by solving the global balance equations of the DTMC for the optimal policy for 


the truncated MDP used in Figure 5.6 


We now present our observations about the nature of 7r((?) in the regime JR using the above example. 
We note that the intuition about the shape of TT{q) in the regime JR, which we discuss below, holds 



Figure 5.10: PMF of queue length for optimal policies in the regime JR for Case 2 for the system in Section 


5.1.4 n = {0.1,1} and 7r//(0.1) = 0.6; A = 0.78 and c(0.78) = 1.0717. 
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Figure 5.11: PMF of queue length for optimal policies in the regime 5ft for Case 3 for the system in Section 
n = {0.1,1} and TTHiO.l) = 0.6; A = 0.8 and c(0.8) = 1.1071. 


5.1.4 


for any sequence of feasible admissible policies, rather than just the sequence of optimal admissible 
policies. 

For l-model, let Qh = {q ■ G [s/ — ey, Su + ey]}, where ey is u}{V) as 17 {, 0. Then it can be 
shown that Pr {Q G Qh] t 1 as 17 {, 0, i.e., as 17 {, 0, the service rates have to be chosen from the 
set [si — ev,Su + ey] or [A — ey, A + ey] for Cases 2 and 3 respectively. 

The expected drift of the queue length when the queue length is (7 is E [Q[m + 1] — Q[m]|Q[m] = q]. 
We note that for admissible policies the expected drift is a non-increasing function of q. 


We consider Case 2 first. We note that since service rates can be chosen from the set [s/—ey, s^-Fey] 
as 17 I 0, intuitively the expected drift in the queue length for small queue lengths in Qh is positive, 
while for large queue lengths in Qh the expected drift is negative. From the geometric upper bound 
on 7^{q), intuitively 7 r(q) grows geometrically for small queue lengths. Since the expected drift of 
the queue length for large queue lengths is negative, from the geometrically decreasing upper bound 
on Pr {Q > g — 1} and geometrically decreasing lower bound on Pr {Q > q] from [51 Theorem 
3] we have that 7r(g) decreases geometrically to zero for large enough q. We note that this is the 


geometrically increasing and decreasing behaviour of 7 r(g) which is illustrated in Figure 5.10 We 
also note that this behaviour holds for any 17 > 0 and as illustrated in the example in above leads 
to log (y) behaviour of the average queue length for any sequence of admissible policies. 


For Case 3, we note that for q G Qh, as 17 {, 0, the expected drift approaches 0. Suppose we define 
qa as largest queue length in Qh and ey as al7 where a > 0 is a constant. Then it can be shown 
that for any q ^ Qh 

Pr{Q < q] < ^^ X a constant. 

We note that the drift d = ey = aV. Hence, Pr {Q < q] ^ O ■ Therefore, using the bounds 

on the stationary probability distribution, we are able to obtain the intuition that the stationary 
probability for any g G Q/j is constant (this is illustrated in Figure [STT]) and 0(17). Then we have 
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that the largest q such that Pr {Q < q} < ^ \s Q (y), and therefore so is the asymptotic lower 
bound on the minimum average queue length. 

For R-model, we let Qh = {q ■ G [A — ey, A + ey]}- The intuition for the shape of the distri¬ 
bution for R-model is similar to that for Case 3 above. With ey = aVV the stationary probability 

1 

W 

is the asymptotic lower bound on the minimum average queue length. 


^ and therefore so 


for any q € Qh ts O [ Vv) and thus the asymptotic lower bound on <7 is 0 


5.6 Asymptotic lower bounds for ergodic {A[m]) and {H[m]) 


In this section, we present an asymptotic lower bound for the optimal value of (5.4) when {A[m]) 
and {H[m]) are ergodic processes, for l-model. 

We restrict to policies in Fs, which choose a batch service size 5[m] = S{Q[m — l],H[m]) as a 
function of the current queue length and fade state and independently of anything else. We note 
that such policies may not be optimal. The development for ergodic {A[m]) and {H[m]) closely 


follows the development in Section 4.3.7 


Since (A[m]) is assumed to be ergodic, we have that almost surely 


1 


M 




M^oo M 

m=l 

and A < Smax, where Smax is the largest batch size which can be served, as defined before. We 
also assume that the arrival process {A[m]) is such that 

NAl : Let a[m- 1] = (Q[0] = = ai,Q[l] = qi,A[2] = a 2 , ■ ■ ■, A[m - 1] = am-i,Q[m - 

1] = qm-i)- We assume that 


inf min Pr {^[m] = 0\a[m — 1]} = t'a > 0, 
mez+ {ai,...,am-l} 

—1} 


NA2 : {A[m]) is independent of {H[m]). 

Since {H[m]) is assumed to be ergodic, we have that 

Pr{H[m] = h} = 7 r//(/i), V/i G > 1, 


with = 1. 

We restrict to policies 7 G F^ for which the following limits exist 

lim Pr {Q[m — 1] = ( 7 , F7[m] =/i} = ttq /i),Vq G and/i G 77, (5-8) 

m—^oo ’ 
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with '^q h'^iQ^h) = 1. We note that for such a policy the following limits also exist 


^ M-l 

77 y] = 7r(g),VgGZ+, 

M -^00 M 

m=0 

1 " 

77 = ^)'S'M = s} = vr//s(/l, s),Vs G {0, . . . ,S'maa:} ,v/l G 

M^oo IVl 

m=l 

1 ^ 

77 y]= 4 = vr^(s),Vs G {0,... ,5max} • 

M^oo M 

m=l 


We again restrict to a set of admissible policies Fq C F^, which are defined similarly as in Section 


4.3.7 


A policy 7 G Fs is admissible if : 


NGl : the I imits in ( |5.8D exist 

NG2 : 7 is mean rate stable, i.e., Ylh£H^s=o'^ ‘^H,s{h,s)s = A, 

NG3 : the average service rate s{q) = (/i)E5(g, h) is a non-decreasing function of q for 7. 


For any admissible policy 7 , we have that P( 7 ) = J2henYlse{o,...,Sma.}'^f^^s{h, s)P{h, s) and 

<3(7) = Eg ^(9)9- 

For any admissible policy 7 , we have that ^( 7 ) is bounded below by the optimal value of 

minimize J2h&n Ef^J" T^H,sih, s)P{h, s), 
such that Y.hen Ef=o" ^H,s{h, s)s = X, 


where 7^H,s{h, s) is determined by the policy 7. The optimal value of the above problem is bounded 
below by the optimal value of 


minimize EheW ^H{h) Pslh(s)^(h, s), 
such that El/ie-H Ef=o" ^HWPslk(s)s = X, 


which is c(A). We note that in the above optimization problem, the minimization is over all 
possible conditional distributions on the batch size. The conditional distribution Ps|h('S) has the 
interpretation of the fraction of time a batch of size s is used when the fade state is h. 


Let us denote the optimal conditional distribution that achieves c(A) in (5.7) by P*|/j(A),V/i G 77. 
Similar to the approach in Section 4.3.7 we now show that there exists a sequence of admissible 
policies 7 y for a sequence F | 0 such that limyj^o = c(A). We define random variables 

Uv{h) ~ P*|/j(A -F V),yh G 77. For a particular V > 0, the policy 71 /, serves S[m] = min((5[m — 
l],t/y[m]), where for each m, Uv[m] is an independent sample of Uv{H[m]). Then the queue 
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evolution under is 


Q[m + 1] = max — Uv{H[m + 1]), Oj + A[m + 1], with Q[0] = qo,\/m > 0. 

Applying PH Lemma 1] we have that the limit TTg^niQ, h) = limm->-oo Pr {Q[m] = q, H[m + 1] = /i} 
exists. With = max ( go — (^[1])) 0 I, we can define a sequence Q'{m] which evolves as 


Q' [m + 1] = max 


Q'[m] + A[m] - Uv{H[m + 1]), 0 


Vm > 1. 


We note that Q[m] = Q'[m] +^[m],Vm > 1. Since the sequence C,[m] = A[m] — Uv{H[m + 1]) 
is ergodic, EC[1] = —V < 0, and 74[1] ^ A.yyif 2 Q^f we have that lim ^—}-oo P^ ^ oo^ — 1. 

We note that the sequence of policies 'yv is a sequence of admissible policies, with limy^o P{lv) = 
c(A). Therefore c(A) = inf-ygra P{l)- 


Now we obtain the following asymptotic lower bound, the proof of which is similar to that of Lemma 


4.3.23 We note that the quantities si^Su and the line l{s) are defined as in Section 5.3 


Lemma 5.6.1. For an ergodic arrival process {A[m\,m > 1), satisfying NAl and NA2, an ergodic 
fading process {H[m],m > 1), and for any sequence of admissible policies 7 ^ (satisfying NGl, NG2, 
and NG3), with P{'^k) — c(A) = Vfc 0, we have that Qi'jk) = ^ (^log cases 2 and 3. 


Proof. For some positive e < s/, let qs = inf {g : KS{q,H) > si — e}. Then, as in the proof of 
Lemma 


5.3.1 


we have that Uk = c{s{Q)) — c(A) < I 4 . We consider a particular policy 7 in 


the sequence with E^c(s((5)) — c(A) = U. As in the proof of Lemma 4.3.23 with U in place of 
V, we can then show that ^nd Pr {S{q, H) > 0} > ^|^^,Vg > g^, since for 


g > qs, ES{q,H) > si — e. Now we relate the stationary probability 7r{q),q > qs to Ylq=o^ 
We have that for a g > g^ and for every m > 0 


Pr {Q[m + 1] < g|Q[0] = go} = Pr - 5(Q[m], P[m + 1]) + A[m + 1] < g|Q[0] = go} 
Let a[m] = (Pfl], Q[l],..., Q[m - 1], S[m]). 

We have that Pr {Q[m] — S{Q[m],H[m + 1]) + A[m + 1] < g|Q[0] = go} 


= E, 


'(T[m] 


Pr{Q[m] - S{Q[m],H[m + 1]) + A[m + 1] < g|Q[0] = go,fT[m]} 


— Eg-fm' 


Pr{Q[m] = g|Q[0] = go,cr[m]} x 
Pr {S{Q[m], H[m + 1]) > 0|(5[0] = go, cr[m], Q[m] = g} x 
Pr {A[m + 1] = 0|Q[0] = go, a[m\,Q[m] = g, {S{Q[m],H[m + 1]) > 0}} 


199 











We note that since a[m] does not involve the history of the fade process we have that 


Pr {S{q,H[m + 1]) > 0|Q[0] = qo,a[m],Q[m] = g} > 


si-e 

Sm.n.T. 


Since ((5[0] = qo,a[m],Q[m] = q,{S{Q[m]) >0}) = (Q[0] = qo, A[1],Q[1],..., A[m],Q[m] = 

on 


q), using property NAl, we obtain the same lower bound as in the proof of Lemma 4.3.23 


Pr {Q[m + 1] < (7|Q[0] = go}- Following the rest of the steps in the proof of Lemma 4.3.23 
obtain that Qi'jk) = F2 ^log Since Uk < Vk, we have that Qi'jk) = ^ 




we 

□ 


We note that NA2 is not used in the asymptotic lower bound, while it was used in showing that a 
sequence of admissible policies exist for which P( 7 fc) i c(A). 

The above asymptotic lower bound is weak for Case 3, since we have obtained an O (y) asymptotic 
lower bound when {A[m\) is IID. We note that the above asymptotic lower bound can be applied 
to the system considered by Huang and Neely [30j. We note that a similar asymptotic lower bound 
can be derived for the R-model also. 


5.7 Queueing models with admission control 


5.7.1 System model 


We consider the optimal tradeoff of average queue length and average power for a fading point- 
to-point link, when the packets arriving to the link can be dropped, subject to a constraint on the 
utility of the time average throughput of the packets which are transmitted. We indicate only the 


differences from the models in Sections 5.1.2 and 5.1.3 In each slot m, a random number R\m] 
of packets arrive into the system. We assume that {R[m],m > 1) are IID, with i?[l] < Amax, 
Ei?[l] = A, and ?;ar(ii[l]) = rr^ < oo. We also denote the expectation with respect to the 
distribution of i?[l] as E/j. At the end of slot m, A[m] < R[m] packets are admitted into the 
infinite length transmitter queue, while R[m\ — A[m] packets are dropped. The fade state process, 
{H[m],m > 1), is as before. We also assume that the arrival process {R[m]) is independent 
of {H[m]). At the beginning of slot m > 1, Q[m — 1] denotes the number of packets in the 
transmitter queue. The transmitter starts transmission of S'[m] packets at the start of slot m. We 
assume that just before the end of slot m, A[m] < R[m] packets are admitted into the transmitter 
queue. We note that under our assumptions on (^[m]), and go. Qi'm] G M+,Vm > 0. 

For this model, a policy 7 for operation of the transmitter is the sequence of service and arrival 
batch sizes (S[l], ^[1], S'[2], ^[2],...). The set of all policies is denoted as T. If 7 is such that 
S[m -L 1] = S{Q[m], H[m + 1]) and A[m -L 1] = A{Q[m], R[m + l],P[[m -L 1]), where S{q, h) 
and A{q,r,h) are randomized functions, then 7 is a stationary policy. The set of stationary policies 
is denoted by T^. Since {R[m], H[m],m > 1) is assumed to be IID, we have that for a 7 G T^, 
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{Q[m],m > 0) is a Markov chain evolving on M+. 

If we assume that A[m], R[rn],qo, S[rn] G Z+, then the queue evolution Q[m] G Z+ and the 
model is denoted as l-model-U. On the other hand, A[m],R[m],qQ,S[m] G M+, then the queue 
evolution Q[m] G M+, and the model is denoted as R-model-U. Like R-model, R-model-U with a 
strictly convex P{h,s) function is usually used as an approximation for l-model-U. 

We define the average throughput of a policy 7 G as 


^(7wo) = liminf 

M^OO IVl 


M 


_m=l 


m 


Q[0] = qo 


(5.9) 


Let U{a) : [0,^max] —t E+ be a strictly concave and increasing function of a, with f7(0) = 0. 
The utility of transmitting the packets is U{A{'^,qQ)), for a policy 7 . The average power for a 


policy 7 G Ps is P{'y,qo) and the average queue length is <5(7,go), as defined in (5.2) and (5.3) 
respectively. 


The model considered by Neely |44j is the same as R-model-U. It is shown in [44j that there exists 
a sequence of policies 7 ^ G P^ with a corresponding sequence f4 i 0, such that A{'yk,qo) > pA 
{0 < p < 1 ), ( 5 ( 7 fc,go) = O (^log and Pi'jkiQo) is at most Vk more than the minimum 

average power required for queue stability. It is also shown in j44j for \R\ = 1, that if 7 ^ is any 
sequence of policies, with P( 7 fc,go) at most Vk more than the minimum average power required for 
queue stability and A{'yk,qo) > p\ then Q{'yk,qo) = ^ (log (tr)) 14 4 - 0 . 

We consider the optimal tradeoff between ( 5 ( 7 , go) and P’(7,go) subject to the average utility 
U{A{'y,qo)) being at least a positive Uc < U{Amax), for the class of stationary policies P^, for 
l-model-U and R-model-U, in this chapter. The constraint U (^( 7 , go)) > Uc is equivalent to having 
the constraint A( 7 ,go) > U~^{uc), where U~^ is the inverse function of U. We note that since 
the arrival rate is not the same for all 7 G P^, minimization of the average queue length does not 
directly correspond to minimizing the average delay of the packets. Asymptotic bounds on the 
average delay can be derived using Little's law and are discussed in this chapter. 


5.7.2 Problem formulation for l-model-U and R-model-U 


The general tradeoff problem that we consider is 


minimize < 5 ( 7 , go) such that P(7, go) < Pc and A('y,qo) > pX, 
7er 


where 0 < /? < 1. As in Section 4.2.1 we can show that if Pc > c{pX) for l-model-U or if 


Pc > cr{pX) for R-model-U (which are the minimum average powers required for stability while 
supporting an arrival rate of pX rather than A) then there exists an optimal stationary policy 7 * 
with stationary probability vr*. Therefore, we can restrict ourselves to the set of stationary policies. 
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As for l-model and R-model, we consider the above tradeoff problem for a set of admissible policies 
Fq. However, since there is admission control, we relax the irreducibility requirement (for l-model-U 
and R-model-U) as follows. For an admissible policy 7 , the Markov chain (Q[m],m > 0) has a 
single positive recurrent class TZ^ which contains 0. Furthermore, the cumulative expected queue 
cost as well as the cumulative expected power cost starting from any state qo until TZ^ is hit are 
finite. 

We note that for a 7 G Fq, Q{'y, qq) = < 3 ( 7 ), P{j, go) = ^’( 7 ), and ^( 7 , go) = ^( 7 )- A policy 7 is 
defined to be admissible if: (i) the Markov process {Q[m],m > 0) under 7 is aperiodic and positive 
Harris recurrent on a single recurrence class TZ.y with stationary distribution tiQ (ii) ( 5 ( 7 ) < 00 , 
and (iii) s{q) is non-decreasing in q, where s{q) = W) is the average service rate at 

queue length q. We note that for a 7 e 

Q(7) = E^Q, 

P{l) = ■i^h^S\Q,H P{H,S{Q,H)), 

A{'y) = E^E^^Ei^EA((5, i?, W). 

Let the average service rate be 5 ( 7 ), then 5 ( 7 ) = EttEtt^E^iq W). 

The problem TRADEOFF that we consider is 

minimize( 5 ( 7 ) such that ^( 7 ) < Pc and ^( 7 ) > pX. 

'r&Ta 

The optimal value of TRADEOFF is denoted as Q*{Pc, p)- Suppose 7 is feasible for TRADEOFF. 
Then 


S{j) = A{'y) > pX. 


Now we note that P{'y) is bounded below by the optimal value of 


minimize E^E^^E^ig W)), 

such that E^E^^E 5 |q W) > pX. (5.10) 


We note that E„E^^Es\q^hS{Q, H) = E„^E^Es\q^hS{Q, H). We have that E^^E^Es\q^hS{Q, H) 
EttjjEs^hS and E^E^j:^E 5 '|q j:^P(iF, S{Q, H)) = E-,^jjEs\hP{H, S), where the conditional distribu¬ 
tion of S given H depends on the policy 7. Then the optimal value of (5.10) is bounded below by 


^We note that ti{A) = 0 for any A TZ^y 
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the optimal value of 


minimize S), 

such that > p\, (5.11) 


where we minimize over all possible conditional distributions for S given h, irrespective of the policy 
7 . We note that for l-model-U, these distributions have support on {0, ..., whereas for 

R-model-U they have support on [0, S'maa;]- 


We note that ( |5.11 ) has feasible solutions only if pX < Smax- The optimal value of the above 
problem is c{pX) for l-model-U and cr{pX) for R-model-U, since the constraint is satisfied with 
equality So, we have that for 7 G r^, P{p) > c{pX) for l-model-U and P{'y) > cr{pX) 
for R-model-U. Thus, TRADEOFF has feasible solutions only if Pc > c{pX) for l-model-U and 
Pc > cr{pX) for R-model-U. 


We now show that c{pX) and cr{pX) are both infj^.^gp^ A( 7 )>pA} Pil) l-model-U and R-model- 
U respectively. For l-model-U, we consider a sequence of policies 7 ^, where for each 7 ^, at each slot 
m, each customer in the batch R[m] is admitted with probability p and dropped with probability 
I — p. Then V 7 fc we have that A{'^k) > pA. Now as for l-model, 7 ^ is such that P{'yk) = c{pX) + Vk 
and Qi'jk) = d Thus, if pX < Smax, we have that there exists a sequence of admissible 

policies 7 fc, such that P{'yk) = c{pX) + I 4 , ^( 7 ^) > pX, and Qi'jk) = O for 3 sequence 

14 i 0. For R-model-U, we choose A[m] = pR[m], and then as for R-model, serve the customers 
using a sequence of policies such that Pi'jk) = cr{pX) + I 4 arid Qi'jk) = O Flence, c(/?A) 

and CRipX) are infj^.^gp^ ^( 7 ) for l-model-U and R-model-U respectively. 

In the following, we obtain an asymptotic characterization of Q*iPc,p) in the asymptotic regimes JR 
as Pc i cipX) for l-model-U and Pc | CRipX) for R-model-U, under the assumption that pX < Smax- 
We recall that c(s) is a non-decreasing, piecewise linear, and convex function of s G [0,5maa;], 
whereas CRis) is a non-decreasing strictly convex function of s G [0,5maa;], with c(0) and cr(0) 
both being 0. 


5.7.3 Asymptotic bounds 

We first obtain an asymptotic lower bound for R-model-U. We then outline the derivation of the 
asymptotic lower bound for l-model-U, since it can be obtained using very similar techniques as for 
R-model-U and as in Section IsTSl 

We assume that i2[l] satisfies the property: 


^If the distribution which achieves the minimum in (5.111 is such that E.,rifEs|£rS > pX, then it is possible to 
show that there exists another distribution which has a strictly smaller E^^Es\HPiH,S). 
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RA* : Pr < ^} = > 0, for some A such that 0 < A < pX. 

Let 7 be an admissible policy with P{'y) — cr{pX) = V and ^( 7 ) > pX. For the policy 7 , let 
qi = sup {q : s{q) < pX — e}, for an e chosen such that 0 < e < pA — A. We note that the average 
power used when the queue length is q is S{q, H)), which is bounded below by the 

optimal value of 


minimize 
such that 


E^„Es\hP{H,S), 
^tth^S\hS > siq), 


(5.12) 


where we minimize over all possible conditional distributions for S' given h, irrespective of 7. We 
note that problem (5.12) is the same as ( |5.11 ) except that the constraint is s{q) instead of pX. 
Therefore S{q, H)) > CR{s{q)). Since E^E^jjEs\q^hP{H, S{Q, H)) = P{j), we 

have that P{'y) > E.,^cr{s{Q)). Therefore, E 7 rCH(s(Q)) — cr{pX) < Pij) — cr{pX) < V. Since 
cr(.) is a strictly convex and non-decreasing function, we assume that the second derivative of 
cr(s) is positive at s = pX. As S('y) = E.^s(Q) > pX, we have that 


IEttCkCsCQ)) = IEti 


cr{pX) + 


dcR{x) 


dx 


{s{Q)-pX) + G{s{Q)-pX) 


p\ 


where G{x) is a strictly convex function as in [71 eq (41)], with G(0) = 0 and U=o = 0. Since 
E 7 rs(Q) > pX we have that 


E^Cij(s(Q)) - cr{pX) > E^G{s{Q) - pX). 


Thus, we have that E 7 rG(s(Q) — pX) < V. Therefore, for qi as defined before. 


rqi 


G{s{q) — pX)d7r{q) < V. 


Since G{.) is strictly convex, for a positive oi we have that 


r-<?i 


1 2 


{s{q) - pX)d-K{q) 


V 

< 

ai 


where ai > 0. Since for q < gi, s{q) < pX — e, we have that 

Pr{Q<qi}< ^ 


aie 


2 • 


(5.13) 


Let S(q) be the random service batch size when in state q. We have that Pr{S{q) > s} = 
ff^Pr{S{q,h) > sjdTTnih). 

Lemma 5.7.1. For 7, for A as in RA*, with qi defined as above, we have that infg>,jj Pr {S{q) > A} > 
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6s > 0, where (5s = 


pX—e—A 


Smax A 


Proof. We note that qi = sup{g : s{q) < pX — e}, for 0 < e < pX — A, where A is as in RA*. Let 
P{S{q)) denote the distribution of S{q). We note that by definition, Vg > qi, 


s{q) > pX-€, or, 
sdP{S{q)) > pX — e. 


(5.14) 


Then, 


SmaxdP{S{q)) 

> 

pX - e, 


Pr{Siq) > A} 

> 

pX - e, 

or, 

Pr{S{q) > A} 

> 

pX — e 
Smax 

-A 

-A 


Thus for any q > qi, Pr{S{q) > A} > (5s > 0, where 6s = ^-a - ^ 

From the above result we have that qi > A. 

Lemma 5.7.2. For any sequence of admissible policies 7 ^ such that ^( 7 ^) > pX and P{^k) — 
cnipX) = 14 i 0, we have that Q{jk) = (log 


Proof Let us consider a particular policy 7 in the sequence 7 ^ with 14 = V. As 7 is admissible we 
have that 

roo 

Pr{0<Q<qi}= P{q,[0,qi))dTr{q), 

Jo 

where P{q, Q) is the transition kernel of the Markov chain. Hence, we have that 

/■gi+f 

Pr{f)<Q<qi} > / P{q,[f),qi))d-K{q), 

Jqi 

A 


> eJsPr {qi<Q<qi + — }, 


where we have used Lemma 5.7.1 and the property RA* to lower bound Pr {Q[m + 1] < qi\Q[m 
by e(j(5s. Let pa = e'^ds- Also, for any q' > q, let us denote Pr {q < Q < q'} by 7r[q,q'). 


] = ?} 


Then we have obtained that 


Similarly, we have that 


TT 


7r[0,gi) > Pair 


0, gi + — ) > Pair 


A 

+ ^ ) • 


A A 

(Zi + -,gi+2- 
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which can be written as 


7r[0, g^i) +7r 

7r[0,gi) 


91 + ^1 > PaTT 


1 + 


pa 


By induction, for m > 0, we have that 


Pa V Pa 


> /OaTT 


A A\ 
+ 2 — ) , 


A A 
+ -,(?! + 2- 


> vr 


A , ,A\ 

qi+'m — ,qi + (m+ 1)— ) . 


Hence, we have that for m > 0, 

A' 


vr 


+ ) = 


^ [91 + + (^ + i)t) > if "1 > 0 , 

0 , if m = 0 , 


< 


7r[0,gi) 


1 + — - 1 
pa ' 


pa 


1 + ^-1 


= 7r[0,gi) 


1 + -) -1 
pa 


Since Pr {Q < gi} < (from (5.13)), if m is the largest integer such that 


TT [0,qi) + vr 


A\ 1 
qi,Qi + " 1 ^ ] - 2 ’ 


then ( 5 ( 7 ) > Suppose mi is the largest integer such that 


7r[0,gi) + 7r[0,gi) 


1 

1 + — I -1 

Pa 


1 \ mi 


Then mi < m. Using (5.13), if m2 is the largest integer such that 

1\”^^ aie2 

1 + — < or , 

PaJ 2\/ 


m2 < log/ 1 


V 21" 


aie 


then m 2 < mi. We have that 


m 2 = 


log/i , 1 


aie 


1+^t) V 2 y 


Since Q{-f) > ^ > ^ > we obtain that 


Q(7)>^(log, . (If 1-1 


(5.15) 


(5.16) 
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So for the sequence of policies 7 ^ with 14 | 0 we have that Qi'jk) = ^ (^log ■ 


□ 


We now outline the derivation of an asymptotic lower bound for l-model-U in the regime 5R. The 


analysis of l-model-U proceeds in a similar fashion as in Section 5^ the piecewise linear function 
c(A) and the quantities ap,p > 1 are similarly defined. The three cases which then arise are : (1) 
0 < pA < 02, (2), Qp < pX < ap+i,p > 1, and (3) pX = ap,p > 1 . For Cases 2 and 3, proceeding 
similarly as in the proof of Lemma |5.3.1| it is possible to show that, for any sequence of admissible 
policies 7 fc such that 4 ( 7 ^.) > pX and ^( 7 ^)— c(pA) = I 4 | 0, we have that Qi'jk) = ^ (^log (tr)) ■ 
We note that we do not have any asymptotic results for Case 1, although numerically it can be 
shown that Q*(Pc) < 00 even if Pc = c{pX). 


Remark 5.7.3. We note that a similar logarithmic asymptotic lower bound can be obtained for a 
different class of admissible policies. The difference is in the definition of the monotonicity property. 
For this new class of admissible policies, the average drift KS{q,H) - KA{q, R, H) is assumed to 
be monotonically non-increasing in q. 


Remark 5.7.4. We comment on an asymptotic upper bound for TRADEOFF, which is achieved by 
the sequence of Dynamic Packet Dropping (DPD) policies in [A?]- A DPD policy is parametrized 
by the quantities P,e,uj, and q. The policy chooses a batch size sdpd[it^] in each slot where 


SDPD[m\ = Him argmax 

VsCfO,'" ,Smax} 


|x[m- 1] 


_ 11 _ i _ 


I -/3P{H[m],s) ,Q[m-l] j , 


where {X[m]) is a virtual queue which evolves according to 


X[m] = max (X[m — 1] — S£)P£)[m], 0) -F (p -F e)i?[m]. 

For the DPD policy, through admission control, the queue length process {Q[m],m > 0) evolves as 


Q[m] = mm[q,Q[rri-l]-SDPD[rn] +R[rri]], 


We note that A[m] = R[m] whenever Q[m] < q, otherwise only that fraction of R[m] is admitted 
so that Q[m] = q. We note that A{q,r,h) for this DPD policy, is a function only of the current 
queue length q and the current number of arrivals r. 

From Theorem 1 [44], if 0 < a; and ^ ^ B = 

X > ^ ~ ^ sequence of policies 7 obtained by a sequence 

t 00, we have that <5(7) = C>(log(/ 3 )), P{'y) = cr{pX) -F O , and 4(7) > pA. 

We note that DPD policies are not stationary, since each policy depends on an auxiliary state X[m]. 
However, as noted in [44] Section III], using a sequence of admissible policies which are obtained 
from the admissible Positive-Drift Algorithm in [44] by choosing the parameter Q as log (^), where 
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y I 0, it can be shown that the above tradeoff is achievable. 


bound derived in Lemma 5.7.2 is tight. 


Therefore, the asymptotic lower 


5.7.4 Discussion 

Minimization of average delay: When average delay is the performance measure under consid¬ 
eration, then the problem that we are interested in is 

_ _ 

minimize =—- such that P{'y) < Pc and ^( 7 ) > pX. 

7 era ^(7) 

Let the optimal value of the above problem be D*{Pc,p). 

We note that, since c(s) or cji{s) is a convex and non-decreasing function in s G [0, Smax], for any 
admissible policy 7, if 7 is feasible for the above problem, we have that 

E^c(s((5)) < P(7) < Pc, 
c(E,rs(<3)) < Pc, or, 

S{-f) =E^^Q) < c-\Pc), 

where c~^ is the inverse function of c for l-model-L). Consider any sequence Pc^k i c{pX) as A: t 00 . 
Since ^( 7 ) = the objective function in the above optimization problem can be bounded 

above by and bounded below by 7 =^^^. A similar bound can be obtained for R-model-U. 
Then, it follows that the asymptotic behaviour of D*{Pc^k,p) is the same as that of Q*{Pc,k:P) as 
Pc,k i c{pX) for l-model-U and Pc^k i cr{p\) for R-model-U. 


Relation to the asymptotic order optimal tradeoff in [42] : Neely |42| considers a system, 
with both admission control and service rate control, in which the arrival rate A is larger than the 
maximum service rate Smax- The objective is to obtain a sequence of policies 7 ^ which achieve an 
order optimal minimum average queue length Qi'jk) as the average utility U{S{'yk)) approaches the 
maximum utility value Umax = U{Smax)- We note that there is no cost associated with the service 
of packets in [42j. It is shown that for any sequence of policies 7 ^, such that Umax — U{S{'jk)) = 




Vk i 0, Qi'jk) = XI ^log(^^^y A sequence of policies 7 fc such that < 3 ( 7 ^) = O 
and U{S{'jk)) = Umax — Vk is also obtained. We note that as the utility function is assumed to 
be strictly concave and increasing, the throughput value that maximizes the utility is Smax itself. 
For any sequence 7 ^, if U{'yk) t Umax = U{Smax) it can be shown that the probability of using 
a service rate less than Smax decreases to zero. That is, with qi = sup{g : s{q) < Smax — e}, 


Pr {Q < gi} j, 0. Therefore, the proof of Lemma 5.7.2 can be applied to obtain an alternate proof 
for the asymptotic logarithmic lower bound on the average queue length obtained in [42], but for 
admissible policies. 
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Asymptotic bounds for a model with just admission control: We consider a queueing model 
in which there is no service batch size control and no service cost. The queue evolution is assumed 
to be as follows: 


Q[m + 1] = max((5[m] — Sb, 0) + A[m + 1 ], 


where A[m + 1] < R[m + 1] and Sb < Smax is a fixed batch size. The queue evolution is assumed 
to be on Z+. We assume that Ei2[l] = X > Sb- We consider the case with a single fade state. 
We consider the tradeoff of average queue length and average throughput in the asymptotic regime 
where the average throughput approaches its maximum value Sb- 


As in the previous sections, we consider this tradeoff problem for a class of admissible policies r^. 
An admissible policy for this problem is a stationary policy which is stable (as defined in Section 


5.7.2). However, we note that there is no service batch size control, and it is assumed that the 


average admitted rate at a queue length q, a{q) = '^A{q, R) is a non-increasing function of q. The 
tradeoff problem that we consider is: 


minimize ( 5 ( 7 ) such that ^( 7 ) > tc- 

'T&ra 


(5.17) 


We note for any admissible policy 7 , ^( 7 ) < Sb- Consider a sequence of policies jk defined as 
follows. A policy 7 ^, in slot m, admits a packet from the batch of size R[m] with probability pk or 
rejects the packet with probability l — pk- Let the sequence pk = , for a sequence Vk i 0 . Since 

A( 7 fc) = pkX < Sb we have that ■^k is a sequence of admissible policies with ^( 7 *;) t Sfe- Therefore, 
we have that Sb = sup..j,gp_^ ^(t)- We consider the tradeoff problem (5.17) in the asymptotic regime 
3? where tc t Sb- 


We have the following asymptotic lower bound. 


Lemma 5.7.5. For any sequence of admissible policies 7 ^, with Sb — A{'yk) = 14 j, 0, we have that 
Qilk) = ^ (log (t^))- 


Proof. Consider a policy 7 in the sequence 7 ^ with Sb — ^( 7 ) = V. Since 5 '( 7 ) = ^( 7 ), we have 
that 


{sb - l)Pr {s((5) / Sfe} -F SbPr {s((5) = Sfe} > Sb - V, or, 
Pr{s{Q) / Sfe} = Pr{s{Q) < s;,} < V. 


We note that s{q) = mm(g, sj,). 

We now proceed as in the proof of Lemma 4.3.8[ but with the following changes. We define qs to 
be Sb- Then, we have that Pr {Q < = Pr {s{Q) < s;,} < V. 
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Now we obtain a geometric upper bound on 7r(g) for q > qg as in the proof of Lemma 4.3.5 We 
note that for q > qs, wb have that 


Pr {Q[m + 1] < q\Q[m] = q} > Pr {A[l] = 0} , 


since s{q) = Sb- Therefore, with pd redefined to be just Pr {A[l] = 0} in the proof of Lemma 

k 

we obtain that TT{q) < Pr {Q < qg} ^ for q = qg k and A: > 0. Now proceeding as in the 


4.3.5 


proof of Lemmas 


4.3.5 


and 


4.3.8 


^ Pd ^ ° 

we have that for the sequence 7^, Qi'jk) = ^ (^log ^ 


We note that this model is the discrete time equivalent of INTERVAL-ACHOICE. The problem 
considered here corresponds to INTERVAL-ACHOICE-2-1. 


5.8 Single hop networks 

In this section, we illustrate how asymptotic lower bounds for the tradeoff of average power and total 
average queue length can be derived for a system with N source destination pairs communicating 
over single hop links. For example, this could be a user multiple access or N user broadcast 
channel. The model and the associated tradeoff problem that we consider is motivated by Neely 
[43|, who considered the problem of optimally trading off average power with average delay for a 
wireless downlink system, with no admission control. 

5.8.1 System model 

We first consider a model with real-valued queue evolution. We note that the model for the sin¬ 
gle hop network, with no admission control, is a straightforward extension of R-model in Section 


link, n G A^}. We assume that An[m] < Amax,^n. The arrival processes to different 

links are assumed to be independent of each other. The link is subjected to an IID fad¬ 
ing process {Hn[m],m > 1). The fading processes are assumed to be independent across links. 
The queue length at the start of the slot for the link is denoted as Qnifn — 1]. In the 
following, we use the notation X to denote the vector (Xi, X 2 , ■ " ,^n)- The arrival rate vec¬ 
tor is A = (EAi[l], EA2[1], • • • ,EA.7v[1]). The fade state is assumed to take values in PL, with 
min{min„g|i {^n} ^ > 0- The distribution of the fade state is denoted as tth and 

the expectation with respect to this distribution as E^^^. We assume that |'?i| < 00. 

In each slot m, a service batch size vector S[m] is chosen as a randomized vector function S{Q[m — 
l],H[m]) of the current queue length vector Q[m — 1] and the current fade state vector H[m]. 
We assume that Sn[Tn] < Smax,^n,m. A policy 7 is the choice of the randomized function 


5.1.3 


We assume that there is an IID arrival process {A„ 


\m\,m 


> 1) to the queue for the 


n 


th 
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S{q, h),\/q, h. The evolution of the N queues under 7 is given by: 


Q[m + 1] = Q[m] — 5[m + 1] + A[m + 1], 


(5.18) 


with Q[0] = Qq. We note that the process {Q[m]) is a Markov process evolving on The use 
of the service vector s incurs a power P{h, s) when the fade state is h. Similar to the properties 
(Cl) and (C2), we assume that for every h, P{h,0) = 0 and P{h,s) is a strictly convex function 
of s, Vs G [0, Smax]^■ These assumptions can be motivated by the properties of P{h, s) obtained 
in O Chapter 7]. The average power P{'y,qQ) and total average queue length ( 5 ( 7 , 99 ) defined 
as: 




M 


Y,PiH[m],S[m]) 


_m=l 


Q[0] = Qo 


< 5 ( 7 ,9o) 


A 


lim — E 
M^oo M 


'M-l N 

_m=0 n=l 


<5[0] = Qo 


5.8.2 Problem formulation 


We study the tradeoff of ^( 7 , 99 ) with <5(7; 9o) ^ restricted set Tq of admissible policies. A 

policy 7 is admissible if (i) it is stable with stationary distribution 7r(9) (where stability is defined 


similarly as in Section 5.2), and (ii) instead of property G2, 7 is such that 


MG2 : the average service rate for queue re as a function of its queue length, Sn{q) is non-decreasing 
in q. 


For example, for At = 2 and for the first queue, for an admissible policy 7 we require that si((?i) = 
EQ 2 |q^E^^E 5 ^|g^ q 2 jfS'i is non-decreasing in qi. We note that for a 7 G Ta, P{'y,qo) = ^( 7 ) and 
(5(7,9o) = (5(7). As for the single link case, we study TRADEOFF for the single hop network, 
defined as: 

minimize ( 5 ( 7 ), such that P{'y) < Pc- 

7sra 

We define the function cji{X) as the optimal value of 

minimize , S), (5-19) 

such that ETT^E^ijfS = A. 

We note that cr{X) is similar to cr{X) for the single link case. From [43|, we have that cr{X) = 
inf.ygra-^( 7 ), and cre(A) is a strictly convex function of A, for A G [OjSmax)^, with cre(O) = 0. 
We now derive an asymptotic lower bound on the average queue length (5(7fc) for any sequence 
of admissible policies 7 ^ for which P{'yk) — cr{X) | 0. We note that the multiuser Berry-Gallager 
asymptotic lower bound @31 Theorem 2], is rederived in the following, with the extra assumption 
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MG2. The method of derivation illustrates how the asymptotic lower bounding technique can be 
extended to network scenarios. For ease of exposition, the asymptotic lower bound is derived for 
the case N = 2, but can be extended to any finite N. 


5.8.3 Asymptotic lower bound 

We assume that An[l] satisfies 


MAI : Pr{An[l] - S^ax > Sn,a} > en,a, for some positive 6n,a and en,a, Vn. 


We first obtain a lower bound on the marginal stationary probability Pr {Qi > g} = d7r(qi, q 2 ) 

of the two-dimensional Markov process, which is similar to Lemma 4.4. 1[ The lower bound on the 
marginal stationary probability is obtained for the first queue, but can be obtained for the second 
queue also by interchanging the indices of the two queues. 


Lemma 5.8.1. Let {Q[m],m > 0) be as in (5.18), evolving on M^, with stationary probability tt, 
for a 7 G T^. Suppose there exists a ^ such that 


Vgi G [0,5i,d],IEQ2. 


H 


E 


Qi[m + 1] - Qi[m] 


Q 2 [m],H[m + l],Qi[m] = qi 


> —d, 


where d is positive. Then for any gi. A: > 0, A > 0, <5 > 0, A -|- <5 < 5i^a, and 0 < gi -L /cA < qi^d, 


Pr {Qi > qi + kA} > 


Sei,, 


^ei,a + d 

1 - 


Pr{Qi > qi} 

X, \ r 1 /■°° 

oei.a + 0/ J [ d 

where 7r(gi) is the marginal probability distribution of qi. 


The proof is very similar to that of Lemma [4.4. 1| and is presented in Appendix |5.C[ Utilizing the 
above lower bound, we obtain the following result which is the extension of Lemma [5.4.1 to single 
hop networks. As for the single link case, we first express the average power P{'y) in terms of the 
function cr{s). We note that the average power used when the queue length vector is q is bounded 
below by the optimal value of 


minimize E^jjE5| hP{H,S), 

such that E^^Es\hS = (si(gi), 52 ( 92 ))- 

By definition, the optimal value of the above problem is Ci{(si(9i), 52 ( 92 ))- Then, we note that 
^nCR{si{Qi),S 2 {Q 2 )) < Furthermore, since cr{.) is convex, E^cr{si{Qi),S 2 {Q 2 )) > 

crW- 
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Proposition 5.8.2. For any sequence of admissible policies 7 ^ with P{'yk) — c_r(Ai, A 2 ) = 14 | 0, 
we have that Qi'jk) = ( 7 ^)' 


Proof. We note that for the sequence 7 fc, since P( 7 fc)—Cii;(A) = I 4 , we have that E 7 rCi?(si(( 5 i), S 2 (Q 2 ))- 
cr{X) = Uk i 0 , with Uk < 14. Consider a particular policy 7 in the sequence 7^ with Uk = U. 

Let qi^d = supjgi : si(( 7 i) < Ai + eu}, where eu will be chosen later. From the admissibility of 7 
we have that \/q < qi^d, si( 9 i) < Ai + eu. Assume that qd < 00 . Using d = et/ in Lemma 5.8.1 
we have for a 0 < A + <5 < ( 5 a, ^ > 1, and qi = kiX < qi^d 


Pr{Qi>qi }>(—^^—) + ) Pr{Qi>qi,d} + —l - Si{q 

\oei,a + euJ \oei,a + euJ 


i))dTT{qi) 


Or we have that 


Pr{Qi <qi} < ( 1 - ) ] 1-—/ {Xi - si{qi))d7r{qi) 

\ \deua + euj I 


where we have used the non-negativity of Pr {Qi > qud}- Let Dt = ^ J^Jsi{qi) - X)dTr{q). For 
7 we have that E 7 rCij(si(( 5 i), S 2 (Q 2 )) — c_r(A) = U. Let l{si, S 2 ) be the tangent plane to cr{si, S 2 ) 
at (Ai,A 2 ). Then U = E^[cr{si{Qi),S 2 {Q 2 )) - l{si{Qi),S 2 {Q 2 ))], smce EJ{si{Qi),S 2 {Q 2 )) = 
cr{\). Let G{xi,X 2 ) = cr(xi,X 2 ) — l{xi,X 2 ). We note that G{xi,X 2 ) is strictly convex in xi 


and X 2 . Also G(0,0) = 0 and ^(0,0) = ^(0,0) = 0. Proceeding as in steps (42)-(46) in [43l 


dG 


dxi 


8x2 


Appendix A] we obtain similarly as in the proof of Lemma 4.4.2 that 


A < 

eu V ai 


(5.20) 


for a positive ai. We choose ec/ = 4 \ Let fci be the largest integer such that 


1 - 


(5ei,, 


<5ei,a + Cf/ 


fci' 


I roo 

1-/ (Ai - si(gi))d7r(gi) 


1 

< 

“ 2 


Then Pr{Qi < kiA} < 4 and Q{'j) > The same approach also holds if qd = 00 . 


The rest of the proof is similar to that of Proposition 


4.4.2 


small U. Therefore, Q{jk) = ^ ( 7 ^)’ < 14- 


and we obtain that ki = XI 


□ 


We note that the proof of the above lemma illustrates how the lower bounding technique can be 
applied to a multiqueue case by considering each queue on its own, even though the service vector 
S{Q,H) is chosen as a function of the queue length vector Q. The proof depends on the upper 
bound on the marginal stationary probability of a particular queue, which can be obtained from the 
average drift for that queue, conditioned on its own queue length rather than on the queue length 
vector. 
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5.8.4 Discussion 



Figure 5.12: Illustration of c(Ai,A 2 ) as a function of the arrival rates Ai and A 2 . 


Model with integer valued queue evolution: We note that a single hop queueing network 
model with N users can be set up similarly as above, where {A[m],m > 1), {S[m],m > 1), and 
{Q[m],m > 0) are assumed to evolve on Z^. Similar to the function c(A) defined for the single link 
case, a function c(A) (which is the counterpart of cr(A)) can be obtained as the optimal solution 
of the problem ( |5.19 ), but with the conditional distributions of service batch sizes having support 
on {0,1, • • • , Smax}- We note that c(A) has a polyhedral structure [431 Section VII] (we recall that 
c(A) was piecewise linear). Then several cases may arise, e.g., for = 2 we have the following 
cases, which are illustrated in Figure 5.12| : (1) A lies on the interior of a face of c(Ai, A 2 ) of type 
FI, which includes 0 (which corresponds to case 1 for the single link case), (2) A lies on the interior 
of a face of c(Ai, A 2 ) of type F2, not including 0 (which corresponds to Case 2 for the single link 
case), (3) A lies on a vertex V (not 0) (which corresponds to Case 3 for the single link case), and 
(4) A lies on an edge, which is (i) of type E2, having a projection on the (Ai,A 2 ) plane which is 
perpendicular to the A 2 (or Ai) axis or (ii) of type El, having a projection on the (Ai,A 2 ) plane 
which is not perpendicular to either the Ai or A 2 axes. 


We again consider the TRADEOFF problem for this model in the asymptote of small V, where V is 


the difference between Pc and c(A). As illustrated in the proof of Proposition 5.8.2 the asymptotic 
lower bound for the total average queue length can be obtained by separate lower bounds on the 
average queue lengths of the individual queues. Therefore, we expect that the asymptotic lower 


bounds for Cases 2 and 3 can be obtained from straightforward extensions of Lemmas 5.3.1 and 


5.3.5 respectively. As for the single-link case, we do not have any analytical results for Case 1. 


For case 4(i) we expect that the total average queue length will grow as 12 (y) since the probability 
of the average service rate S 2 (Q) being not equal to the arrival rate A 2 should go to zero as V j, 0. 
We note that in case 4(i) the average queue length of the first queue is expected to grow as 
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Q (log (^)) since the probability of the average service rate si(( 5 ) being greater than the arrival 
rate Ai does not go to zero in the asymptote of small V. We note that for the case of integer 
valued queue evolution, different queues may have different growth rates in the asymptote of small 

y. 

For case 4(ii) we expect that the total average queue length will grow as Q (log (’^)), since the 
probability of the average service rate si(Q) and S 2 {Q) being greater than Ai and A 2 respectively 
is positive as the average power constraint approaches c(Ai,A 2 ). 


Model with admission control: We now comment on how the asymptotic lower bound can be 
derived for a N user model as above, but with admission control. We consider a model, which is a 
straightforward extension of R-model-U to the N user case. 

We assume that the arrival rate vector into the system is R[m] in slot m, with = A. 

For stationary policies, the number of packets admitted into the queue is a function A{Q[m — 
1], R[m], H[m]). For an admissible policy, defined as above, the average throughput is ^( 7 ) = 
R, H). The problem that we are interested in is: 


minimize< 3 ( 7 ), such that ^( 7 ) < Pc and ^( 7 ) > pX, 

•y^Ta 


for a p < 1. Similar to the single link case, it can be shown that inf.ygra-^( t) = cji{pX). Then 
under the assumption that Vn G A^}, Pr [Rn[l] < > 0 for some A„ such 

that 0 < An < pXn we can proceed as in the proof of Lemma |5.7.2| to prove that '^■kQu = 
n ^log (tr)) sequence of admissible policies jk with P{'^k) — cr{pX) = I 4 0. 

Therefore, the total average queue length Qi'jk) is also Q ^log 


Similar results can be obtained for the case where the queue evolution is assumed to be integer 
valued, whenever (A, c(A)) does not lie on a face which contains (0,0). 


5.9 Conclusions 


We recall that R-model and R-model-U, both with strictly convex P{h, s) functions, are usually used 
as approximations for l-model and l-model-U respectively. However, as in Chapter 4, we find that 
the asymptotic behaviours of Q*{Pc) in the asymptotic regimes 51? are different for the approximate 
models and the original models. 


We note that the R-model suggests that a strictly smaller minimum average power cr(A) is sufficient 

we 


for stability, compared with c(A) for the l-model, for all X ^ ap,p > 1. From Lemma 5.3.5 


observe that the asymptotic behaviour of the minimum average queue length is quite different for 
the l-model and R-model, for A = ap,p > 1, for which cr(A) = c(A). For such a A, for the 
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R-model, from Lemma 


5.4.1 


we have that the minimum average queue length is L2 ((^) 


the average power is V more than the minimum average power cr(A) and (b) P{h,s) is a strictly 
convex function of s G [0, Smax], foi' every h £ Ti. In contrast, for the l-model, the asymptotic lower 
shows that the minimum average queue length is O (y). We note that the 


bound in Lemma 


5.3.5 


minimum average queue length for the R-model is always a lower bound to the minimum average 
queue length obtained from the l-model, for a given constraint on the average power, but the rate 
of increase of the minimum average queue length as the average power constraint is reduced is 
strictly smaller. 

For Case 1, we find that the Q*{Pc) for R-model (as well as R-model-U) increases to infinity in 
the regime 5R, while for l-model we have numerically illustrated that Q*(c(A)) < oo (therefore, also 
for l-model-U). However, for R-model (as well as R-model-U), if we use a piecewise linear P{h,s) 


function (as in Section 4.4.2), which is the lower convex envelope of the service cost function 
defined for the l-model (as well as l-model-U) on {0,... ,Smax}, then the asymptotic behaviour of 
both the R-model (R-model-U) and l-model (l-model-U) matches. Therefore, a more appropriate 
approximation for l-model (or l-model-U), is a R-model (or R-model-U) with the above piecewise 
linear P{h,s) function. 

provides the asymptotic lower bound to the O (log (^)) upper bound 


We note that Lemma 


5.3.1 


observed by Neely in (431 Corollary 2] for the l-model. This asymptotic lower bound was earlier 
shown only for a specific example ([431 Section Vll-A]). We note that if {A[m]) and {H[m]) are 
ergodic sequences, independent of each other, then for Cases 2 and 3 for the l-model, we have 
obtained a (log (’^)) asymptotic lower bound on the minimum average queue length in Lemma 


5.6.1 As stated, for Case 1, if \1-L\ > 1, we do not have tight asymptotic upper or lower bounds 


on Q*{Pc), as Pc i c(A). Lemma 5.7.2 provides an asymptotic lower bound for R-model-U even 
if |?7| > 1, for admissible policies, while earlier an asymptotic lower bound was obtained only for 
the case |77| = 1. We also illustrate how the asymptotic lower bound can be obtained for a N user 
single hop network and identify a case in which average queue lengths for different queues can have 
different asymptotic behaviours for integer valued queue evolution. 
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Appendices 

5.A The function c(A) in Section 5.2 


In this section, we show that the function c(A) defined as the optimal value of 

minimize S), 

such that = A, 

is piecewise linear if |?^| is finite. Let be the conditional distribution of the batch size s given 
the fade state h. Then the above problem can be written as 


minimize E T^nih) ^ Ps\hP{Ks), 

h s=0 

Smax 

such that ^7rH(/i) ^ = A, 

h s=0 

Smax 

^ and > 0,Vs,/i, 


s=0 


which is a linear program in the variables Ps\h- 

For ease of exposition, in the following we consider the case where |?f| = 2, but the approach 
holds for any finite |?^|. Let s = (0, 1, ..., S’^ax), P{hi,s) = 0), ..., Smax)), Ps\h = 

{Po\hi ■ ■ ■ iPSmax\h)’ and let 1 be a row vector of all ones and 0 a row vector of all zeros, both of 
size Smax- Then the above linear program can be written as 


minimize [7rH{hi)P{hi, s),7rH{h2)P{h2, s)] 


such that 


pi\h, 
L pi\h. 


-KH{hl)s TTH{h2)s 


t 


’ A ’ 

1 0 


Ps\h, 

4- 

= 

1 

0 1 


L Ps\h, \ 


1 


and > 0,Vs,/i. 

We note that the dual of this problem is 


maximize 


[yi,y2,y3] 


such that [yi,y 2 ,y 3 ] 


A 
1 
1 

7rH{hi)s •KH{h2)s 

1 0 

0 1 


< [T^H{hi)P{hi, s),'KH{h2)P{h2, s)], 
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where yi, ?/2, ?/3 £ Let yi{X),y2{X), and y^{X) be any optimizers for the dual for A G [ 0 , Smax]- 
Since c(A) is convex in A (t43j), it is differentiable at all A G [0,S'maa:] except for A G P, where 
V is at most countable [20]. Now consider c(A) for a A 0 V. Then we have that is well 
defined. Furthermore, since strong duality holds for the linear programs above, we also have that 
= —yj|‘(A) [H Section 5.6]. Therefore, for A 0 P, yl{X) is unique. 

We note that the constraint set in the dual problem does not depend on A and has finite number of 
vertices. Therefore, there are only finitely many ways in which yi(A) can be unique. Hence, 
can take only finitely many values. Since c(A) is also non-decreasing in A, we have that c(A) is 
piecewise linear. We note that the above approach generalizes to any finite 71. 


5.B Proof of Lemma 15.3.2 


Let L{q) = be a Lyapunov function. Since for the policy 7, the batch size S{q) could be 

more than q, the queue evolution equation under 7 is written as 

Q[m -F 1] = max(Q[m] — S{Q[m]), 0) -F A[m -F 1]. 

The expected Lyapunov drift is 

A{q) = E [L{Q[m + 1]) - L{Q[m])\Q[m] = q] . 


We note that the randomness in S{q) arises from both the randomness in the fade state as well 
as the randomization of the batch size. The expectation of S{q) is therefore with respect to this 


distribution. Proceeding as in the proof of Lemma 4.G we have that 


A{q) < [(E-§(g) - X) + K 


where K = 




Now by definition, the policy 7 is such that 


ES{q) = 


si, for 0 < g < q^, 
Su, for qv < q. 


Then we have that for q < qv 


A{q) < [A - s, - i^] . 
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And for q > Qv, 


A{q) < [(s„ - A) + i^], 

= [X-si-K]+ [su - sz] , 

< [X- si- K]+uj [s„ - si] , 


Hence, for all q we have that 

A{q) < [X- si- K]+uj [s„ - si] . 

We choose ui such that K < X — si. Proceeding as in the proof of @3] Theorem 3(c)], we have 
that 

^Mqv-Q) < ~ 

- {X-si-KY 

Since > e jQ < Smax\ Pr {Q < Smax}i we therefore have that 

pUlS max [ o cjl 

Pr {Q < Sma.} < (5.21) 

{X- si- K) 


Now we note that 


P(7) = Pr{Q<S^ax}nP{H.S)\Q< Smax] T 

Pr {Smax <Q<qv}E [P{H, 5)|5™a, < Q < q,] + Pr {qy < Q} E [P{H, S)\qy < Q] . 

We note that for 7, for Smax < q < qy, EP{H,S) = c{si) and for q > qy, EP{H,S) = c{su)- 
Furthermore for q < Smax, EP{H, S) < c{si). Hence, we have that 


P( 7 ) < Pr {Q < Smax} c{si) + Pr {Smax <Q<qy}c{si)+Pr{qy<Q}c{su)- 


Proceeding as in the proof of Lemma 4.3.10 we have that 


P(7)-c(A) < [c(sz) + mA - c(A)] Pr {Q < S'max} , 


where m is the slope of l{s). 

Now consider the sequence of policies 7 ^ for which qy = log (^) for a sequence 14 < 1 such that 


4.F.1 


we have 


14 i 0. Then we have that P( 7 fc) — c(A) = 0(14). Furthermore, from Proposition 
that ( 5 ( 7 ) = O ^log (tr))' ^ sequence of admissible policies, since s{q) 

is a non-decreasing function of q and Qi'jk) < 00 . 
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5.C Proof of Lemma 15.8.1 


Proof. Define Qi[m] = max(qi, (5i[m]) and Qi = max{qi,Q). As the policy is admissible and qi 
is finite we have that < oo. Therefore 


E 


' Ql .92 


Qi [m + 1] - Qi[m] Qi[m] = qi,Q 2 [m] = q 2 d7r{qi , 52 ) = 0. 


We split the integral over qi into three terms which leads to : 

rqi — A POO 


0 = / / E 

Jo Jo 

rqi fX) 

+ / / E 

J qi—A J 0 

POO POO 


Qi[m + l]-Qi[m] Qi[m] = qi,Q 2 [m]= q 2 dTr{q 2 \qi)d'K{qi) (5.22) 
Qi[m + 1] - Qi[m] Qi[m] = qi,Q 2 [m] = q 2 d7r{q2\qi)dTT{qi){5.23) 


+ / / E 

J qi Jo 


Qi[rn +1] - Qi[m] Qi[m] = qi,Q 2 [m] = q 2 dTr{q 2 \qi)dTT{qi) (5.24) 


We note that for qi < qi — A, (5.22) > 0. Consider (5.24), for which Qi[m] = Qi[m] and 
Qi[m + 1] > Qi[m + 1], Therefore 


POO 

( |5.24D > / {Xi- si{qi))d'K{qi). 

Jqi 


Using the assumption MGS, the above integral can be further bounded below by 

POO 

-dPr {qi <Qi < qi^d} + / (Ai - Si(gi))cf 7 r(gi). 

Jqi.d 


Consider (5.23), we have that Qi[m + 1] — Qi[m] > 0 for qi G [gi — A, gi]. Hence as in the proof 


of Lemma 4.4.1 we use Markov inequality to lower bound (5.23). 

E 


Qi[m + 1 ] - Qi[m]\Qi[m] = qi,Q2[m] = q2 


> ( 5 Pr|Qi[m + 1 ] - Qi[m] > S Qi[m\ = qi,Q2[m] = q2^ , 

> 5 Pr{Qi[m + 1 ] - Qi[m\ > 5 + A\Qi[m] = gi, (52M = 52} , 


where A > 0 and 5 > 0 are chosen such that A + 5 < q. Thus we obtain that 


(5.23) > Sei^aPrJqi - A < Qi < qi} 


Combining the obtained lower bounds on (5.22), (5.23), and (5.24) we obtain that 


POO 

0 > dei^aPr {qi - A < Qi < gi} - dPr {gi < Qi < gi,d} + / (Ai - si{qi))dTT{qi), 

Jqi.d 


= Sei^aPr {Qi > qi - A} - {d +5ea)Pr{Qi > qi} + dPr{Qi > qi^d} + / (Ai - si(gi))d 7 r(gi). 

Jqi.d 
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Hence 


Pr{Qi>qi} > 


^ei,a 
5ei,a + d' 

1 

<5ei a + d 


Pr {Qi >qi-A} + 

POO 

dPr {Qi > qi^d} + {Xi - Si{qi))d7r{qi) 


By induction, as in the proof of Lemma 4.4.1 we obtain that if A: > 0 and qi + kA < qi^^, then 
Pr{Qi>q, + kA} > Pr{Qi>qi} 


5ei,a + d 


+ 


1 - 


y 

+ dj 


1 r°° 

Pr {Qi > + - / {\i - si{qi))d'K{qi) 

® dqi,d 


□ 
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CHAPTER 6 


On the tradeoff of average error rate and average delay 

for point to point links 


6.1 Introduction 

In this chapter, we consider the transmission of a bursty information source over a noisy point to 
point link. Our objective is to transmit the randomly arriving source message symbols such that the 
message symbols are decoded reliably and with minimum delay. Reliability and delay performance 
are measured by the average symbol error rate and the average symbol delay respectively. Under¬ 
standing the fundamental tradeoff between reliability and delay is significant due to the increasing 
use of cross layer scheduling, for allocation of highly constrained wireless resources, in modern high 
rate communication networks. We assume that the transmitter and receiver use a block code (such 
as a LDPC code) to reliably communicate the message symbols, as in many practical scenarios. 

It is known (24l Chapter 24] that if the average information arrival rate into the system is less than 
the capacity of the channel, then arbitrarily low probability of error can be achieved using block 
coding with long codeword lengths and finite average delay. In this chapter, we first characterize 
how the minimum possible average delay grows when the average error rate is made arbitrarily small. 
However, there are cases where arbitrarily long codewords cannot be used. Then there is a positive 
infimum for achievable error rate. Then for this case, we characterize how the minimum possible 
average delay grows as the average error rate is made arbitrarily close to the above positive infimum 
of achievable error rates, using the techniques discussed in Chapter 4. We note that the service 
cost function in this chapter, which is the expected number of message symbols in error when a 
batch of message symbols is transmitted, turns out to be non-convex unlike the convex service cost 
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functions in the previous chapters. The notation that we use in this chapter is summarized in Tables 
16.H and l6^ 

6.1.1 System model 

The system is assumed to evolve in discrete time units of slots. The slots are indexed by n G 
{1, 2, • • • }. Each slot corresponds to one channel use. We note that these slots may be thought of 
as subslots in the slots for the discrete time models in Chapter 4 (this is why the slots are indexed by 
n rather than m in this chapter). The channel is assumed to be a memoryless channel, with input 
alphabet X and output alphabet 3^. The transition probability function of the channel is denoted 
by Py\x- 

We assume that there is no admission control throughout this chapter. The source generates a 
random number As[n] < Amax of message symbols in each slot n. We assume that As[n] is IID 
with E74 s[1 ] = A and far(^s[l]) = <7^, both of which are finite. Each message symbol is generated 
independently and uniformly from a message alphabet Ai of finite cardinality \Ai\. The message 
symbols are assumed to enter the queue just before the slot boundary and reside in the transmitter 
buffer until they are encoded and transmitted. Transmission of a message symbol is assumed to 
require at least one slot. The transmitter buffer size is assumed to be infinite. 

The transmitter is assumed to use random block coding [221 Chapter 5]. Each symbol of the 
transmitted channel codeword is picked IID from a distribution Q(.) on X. At a decision epoch, 
which occurs at, say the start of slot n, if the transmitter decides to transmit, it uses a random block 
codebook, with codewords generated as above. The codebook is characterized by two parameters, 
the number of message symbols (s) which are encoded and transmitted by the codeword and 
the length (r) of the codeword. Starting from slot n, s message symbols are removed from the 
transmitter queue after r slots. We assume that there is no path delay in the transmission of 
the codeword symbols. The receiver is assumed to decode the s message symbols Jointly by using 
maximum likelihood decoding of the codeword. If the transmitter decides not to transmit, then the 
receiver is made aware of the idle state through the control channel and the transmitter idles for tq 
slots. A decision epoch occurs after every transmission period or idle period. We assume that at 
the end of a slot n, events occur in the following order: a) if the service of a batch of symbols ends 
then the batch is removed from the queue, b) new arrivals in slot n are admitted into the queue, 
and c) the queue length state at the beginning of the next slot is obtained. 

For the message symbol, let Ta^i denote the slot in which the symbol arrives into the transmitter 
queue. Let denote the slot in which the symbol departs from the receiver. We note that 
Trf j is the slot in which transmission of the batch containing the symbol finishes. The delay of 
the symbol in the queue is — Ta,i. Let Ei denote the event that the symbol is in error. 

A policy 7 for operation of the transmitter consists of a sequence {{S[m],T[m]), m >1), where m G 
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Symbol 

Description 

n 

slot index 

As[n] 

random number of message symbol arrivals in slot n 

A 

■^max 

maximum number of message symbol arrivals in a slot 

A, cr^ 

mean and variance of As[n] 

Tf 

input alphabet of a point to point channel 

3^ 

output alphabet of a point to point channel 

Py\x 

channel transition probability function 

Q 

distribution on the channel input symbol G X 

s 

a particular batch size 

T 

a particular transmission duration 

Ta,i 

arrival time of message 

Td,i 

departure time of message 


event that the symbol is in error 

m 

decision epoch index 

S[m] 

service batch size for the transmission 

Smax 

maximum service batch size 

T[m\ 

duration of the transmission 

7 [m] 

= (5[m],T[m]) 

Q[m] 

queue length at decision epoch m 

Qs[n] 

queue length at start of slot n 

A[m\ 

random number of message symbol arrivals between (m — 1)*^ and decision epochs 

r 

set of all policies 

Ts 

set of stationary policies 

Peil) 

average error prob. for policy 7 


average delay for policy 7 

Pe,c 

constraint on the probability of error of message symbols 

Dc 

constraint on the average delay of message symbols 

M 

source message alphabet 

\M\ 

cardinality of Ai 

Nc 

fixed codeword length 

h{q,T) 

holding cost; is qr + 

Ts,/ 

set of all policies with transmission time being a fixed parameter 


set of all policies with transmission time being Nc 


Table 6 . 1 : Notation used in this chapter (part I). 


225 




Symbol 

Description 

Qii) 

average queue length for a policy 7 

Cs{s,t) 

expected number of messages in error for a random block code 

Eoip, Q) 

Gallager's random coding error exponent 

p:{Dc) 

minimum average error probability over F^ under delay constraint Dc 

p:j(Pc) 

minimum average error probability over Tgj under delay constraint Dc 

p:.n,(p>c) 

minimum average error probability over F^ under delay constraint Dc 

^iv.(A) 

minimum achievable average error probability for the system to be stable 

Rc 

cutoff rate of a point to point channel 

C 

capacity of a point to point channel 

nis 

a particular message symbol, G M. 

To 

duration of time the transmitter idles 

t{s) 

transmission time as a function of s 


Table 6 . 2 : Notation used in this chapter (part II). 


{1, 2, 3, • • • }. If 5[m] > 0, then 5[m] is the number of message symbols which start transmission at 
the (m — 1 )*^ decision epoch using a codeword of length T{m). If S[m] = 0, then the system idles 
for T[m] = To slots. Let 7 [m] = {S[m],T[m]). The evolution of the system is illustrated in Figure 


(ii) the initial number of message symbols in the transmitter queue qo, and (iii) the arrival process 
up to the start of the (m — 1)*^ decision epoch. The length of the transmitter queue at a decision 
epoch m is denoted by Q[m]. The queue length at the start of a slot n is denoted by Qs[n — 1]. 
The evolution of the system sampled at the decision epochs, for a policy 7, is given by the following 
equation: 

Q[m + 1] = Q[m] — S[m + 1] + A[m + 1], (6.1) 

where S[m + 1] < Q[m] and A[m\ is the random number of arrivals which have occurred in the 
period between the and (m + 1)^^ decision epochs. We note that A[m\ ~ the 

T[m] convolution of ^s[l]. Let T denote the set of all policies. The class of all stationary policies 
Fs is such that for any 7 G F^, 5[m] = S{Q[m — 1]) and T[m] = T{Q[m — 1]), where S and T 
are functions (possibly randomized) of the queue length q. We note that if 7 G F^, then Q[m] is a 
Markov chain embedded in the random process Qs[n]- 

In the following, we consider two separate models, set up as follows. 

R-model-A : Vm > l,A[m] G [0,Amax], S'fm] G M+, and go £ 

R-model-B : Same as the R-model-A, except that 5[m] G [0, 5max]. and T[m] = Nc, m>l. 

We note that for R-model-A and R-model-B, the queue length evolution {Q[m],m > 0) is on the 


6.1 For every m, 7 [m] can be a randomized function of (i) the history ( 7 ( 1 ], 7 ( 2 ], • • • , 7 [m— 1]), 
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Figure 6.1: Evolution of the system for a block coding scheme 

non-negative real numbers. Furthermore, for R-model-B, the maximum batch size is bounded by 
Smax and the decision epochs occur every Nc slots. Hence, for R-model-B, all codewords are of 
length Nc- 

For R-model-A and R-model-B, for a 7 G F, we define the average delay as 

D{j) = limsup (6.2) 

1^00 ^ 

We also define the average error rate as 

Pe(7) = limsup ( 6 . 3 ) 

where is the indicator function for the event £'*. 

Our problem is then to 


minimize Pp ( 7 ) or minimize 11 ( 7 ) 
such that < Dc such that Pe( 7 ) < Pe,c- 

for every Dc > I or 0 < Pe^c < 1- For an average error rate constraint Pe,c > ^ 77 ^' 
the following optimal solution: transmit all the arrivals in a slot in the succeeding slot, using a 
channel input symbol picked independently of the message symbols, and pick the message symbol 
estimates uniformly at the receiver (maximum likelihood decoding). In this chapter, we analyse the 
above problem, for the class of stationary policies F^, in the asymptotic regime where (i) arbitrarily 
large reliability is required, that is, as Pe,c i 0 for R-model-A, and (ii) Pe,c approaches the minimum 
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probability of error for Nc, when the codeword length r is fixed to be Nc for R-model-B. 


6.1.2 Related work 


We note that for obtaining the average error rate of a policy, we would need to know how the error 
events Ei are related to the scheduling policy 7 which chooses the parameters {S[m],T[m]) for 
the random block codebook at every m > 1. It is intuitive that, for a stationary policy 7 , apart 
from the noise introduced by the channel, Ei is a function only of the size s and the length r of 
the codebook, from which the codeword used to transmit the symbol is selected. Gallager [221 
Chapter 5] has provided an upper bound on the average error probability of a random block code 
as a function of s and r. The same reference also provides a lower bound on the average error 
probability of any block code as a function of s and r. The above upper and lower bounds show 
that the average error probability of the random block code decays exponentially with the block 
length for codeword rates less than the capacity of the channel. The upper bound which is obtained 
as an ensemble average over random block codes (Gallager’s random coding upper bound) and the 
lower bound (sphere packing bound) are found to coincide in the exponent for rates greater than 
the cutoff rate for the channel. So for large values of the block length and for codeword rates 
greater than the cutoff rate, the Gallager random coding upper bound can be used as a reasonable 
approximation for the average error probability of the block codeword. Recently Polyanskiy (49) has 
obtained upper and lower bounds as well as an analytically tractable approximation for the average 
error probability of block codewords as a function of the rate and block length of the codebook. 
The approximation has been observed to be tight even for small values of the block length. We 


provide a detailed review of the results from Gallager [23 in Section 6.2.1 wherein the Gallager 
random coding upper bound is used to approximate the average number of symbols in error, as a 
function of s and r, when random block codewords are used, as in our model. 


Now we provide a survey of prior work on characterizing the tradeoff between reliability and delay 
for point to point channels. Javidi and Swamy [2I| consider a point to point link with a random 
stationary arrival process of bits at rate A into an infinite transmitter buffer. Every slots, a 
batch of s = rNc bits (with zero padding, if required) is encoded and transmitted over the noisy 
channel using a block code of fixed rate r and length Nc- The transmission of a bit is said to 
fail if : 1) the bit is part of a codeword that is decoded in error or 2) the bit is decoded past a 
given delay deadline D. The authors study the probability of a bit transmission failure as a function 
of r,Nc, and D. The analysis is asymptotic in nature - in the regime of large delay deadline D. 
It is assumed that b > 2 and the codeword length Nc = ^ as D 00 . The constant b can be 
interpreted as a parameter controlling the division of the overall delay budget D between queueing 
delay and transmission delay. A large deviations result is used to show that the delay deadline 
violation probability of a bit decays exponentially with the deadline H, as D t 00 . Using Gallager's 
random coding upper bound, it is then shown that bit transmission failure probability also decays 
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exponentially with the exponential rate being the minimum of (1/6) times the Gallager exponent, 
and the delay exponent. The results obtained are in a similar vein as ours, as the exponential decay 
of the error rate is shown with respect to the delay deadline. However, we consider average delay 
as the metric and as we shall see, we also obtain the best decay rate over the class of stationary 
policies, using codewords of fixed length. We also show that scaling the codeword length linearly 
with average delay is optimal, with the scaling factor being 2/3. 

Musy and Telatar li4pj consider a continuous time queueing model with a Poisson message symbol 
arrival process, for a point-to-point link with random block coding. The block coding scheme is 
assumed to be such that the codeword length can be varied as a function of the number of message 
symbols which are jointly encoded in a codeword. The codeword length is chosen such that the 
block error probability for every transmission is at most a fixed constant. The authors obtain upper 
and lower bounds on the minimum average delay for a fixed upper bound on the constant block 
error probability. A joint scheduling-coding scheme for a point to point link, with ARQ, has been 
analysed in [7I| by Swamy and Javidi. The dependence of average error rate on delay has been 
considered for non block coding schemes with Poisson arrivals in [80| by Yoon, and for a periodic 
source in [47j by Negi and Goel. 


6.1.3 Overview 


We formulate the tradeoff problem in Section 6.2 For the class of stationary policies, we express the 
average error rate and average delay, in terms of quantities which are analytically more tractable, in 
the same section. In Section [6.2.1| we present a discussion on error exponents for discrete memoryless 
channels. In this chapter, the Gallager random coding upper bound is used to obtain an upper bound 
for the expected number of message symbols in error in every transmission, which is then used to 
approximate the average error rate of stationary policies. We then consider the asymptotic behaviour 
of the minimum average error rate subject to a constraint on the average delay, for R-model-A in 


Section O We show that the minimum average error rate decays exponentially to zero, as the 
constraint on the average delay increases to infinity. The exponential decay rate is shown to be 
two-thirds of the Gallager random coding exponent. We then comment on the exponential decay 
rate for a queueing model in which the queue length evolution is assumed to be on the set of 
non-negative integers. We also consider the asymptotic behaviour of the average error rate for a 
class of policies with codeword length dependent on the batch size, in the same section. 

The asymptotic behaviour of the minimum average delay subject to a constraint on the average 


error rate is then characterized for R-model-B in Section 6.4 This particular problem is similar 


to the TRADEOFF problem, analysed in Ghapter 4, except that the service cost function is non- 
convex. For R-model-B, as all codewords are of finite length Nc, it is intuitive that the infimum 
of achievable average error rates, for any finite average delay, is bounded away from zero. The 
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non-convex nature of the service cost function leads to the result that depending on the value of 
the arrival rate A, the minimum average delay is either © (log (f))’ the average 

error rate constraint is V more than the above positive infimum of achievable average error rates. 
We then consider the tradeoff problem for a similar model, where the queue length evolution is on 
integers, for which the results from Chapter 4 directly applies. 


6.2 Problem formulation 


In the following we restrict ourselves to a special subset of which is the set of admissible policies. 
A policy 7 G Fs is admissible if: 

1. the embedded Markov chain (EMC) Q[m] is positive Harris recurrent, with stationary distri¬ 
bution TT^, and, 

2. the embedded average T((5)) < oo, where h{q,T) = qr + 

We note that the function h{q,T) can be interpreted as the cumulative expected queue length in 
a deterministic transmission/idle period of duration r, when q message symbols are present in the 
queue at the beginning of that period. Every admissible policy is stable, according to the definition 
in Section 4.1.3| of Chapter 4. To avoid unnecessary notation, and since in the following we consider 
only admissible policies we redefine as the set of all admissible stationary policies. We recall 
that T[m] is the duration of the transmission/idle period beginning at the (m —1)^^ decision epoch. 
The set of all 7 G T^, for which all transmission and idle periods are of a fixed duration Nc, i.e., 

T[m] = tq = Nc, is denoted as Ts tVc- Let Tgj = Lhat when operating with a 

policy 7 G Esj, a transmission duration Nc{'y) is chosen at the first decision epoch, which is kept 

fixed at all the other decision epochs. We have that T D T^ D Fgj D ^s,Nc- We now express 

£>( 7 ) and Pei'y) as averages of functions of the queue length and block coding parameters at the 
decision epochs when 7 G T^. 

The average queue length Qij) for a policy 7 G T^ is defined as[^ 


Q( 7 ) = Jim 

N^OO 


En=0 QM 

N 


(6.4) 


For 7 G Ts, we note that {Q[m],T[m + 1 ]) is a Markov renewal process, with the renewal instants 
corresponding to the decision epochs. In order to express the time average < 5 ( 7 ) in terms of a 
function of Q[m], we use the Markov renewal reward theorem [331 Theorem D.16]. We associate 
a reward R[m] with the renewal cycle. The reward R[m] is the cumulative queue length over 


^We note that the requirement of monotonicity as in the previous chapters is imposed only for R-model-B 
^We note that this is a sample path definition, which is different from the definition in Chapter 4. However, for 
7 € Fs, the two definitions are equivalent. 
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the renewal cycle, with the queue length at the beginning of the cycle being Q[m — 1] and cycle 
length being T[m\. Since Q[m] is Markov and 7 G we have that, conditioned on Q[m — 1] and 
Q[m' — 1], R[m] is independent of for any m! / m. Let T[m] be the number of slots up to the 
start of the renewal cycle (r[l] = 0). We have that the expected reward E [R[m]\Q[m — 1]], 
in cycle m, is 


T[m] -1 n 


Q[m - l]T[m] + 'Y Y^ As[T[m] + n'] 

Q[m - 1] 

n=l n'=l 



Since As[n] are assumed to be I ID we have that 


r[m] = E 


Q[m- l]T[m] + 


T[m]{T[m] - 1)A 


Q[m 



We define the holding cost at a decision epoch, where the queue length is q and a deterministic 
transmission/idle time r is chosen, as a function h{q,T) defined as 


h{q,T) 


qr + 


r(r — 1)A 
2 


We note that r[m] = Kh{q,T{q)), if Q[m — 1] = q, and the expectation is over the randomized 
choice of r. We note that for any 7 G F*, since K.,^^Kh{Q,T{Q)) < 00 , we also have that 
Kt^^KT{Q) < 00 . Then, we have from (33l Theorem D.16] that 

lim ^n=oQM - ^.,^T\QHQ,rm 

N^oo N E,r^E7-|QT(Q) 

Hence, for 7 G F^, using Little's law [78], we have 


D{j) 


Qii) 

A 


^77^^r\QHQ^'T'iQ)) 

AE^^E7-|qT(<5) 


(6.5) 


To express Pei'y) as a time average we use the generalized H = XG form of Little’s law [78]. Define 
Fi[n] = 0 for n G {0, ... ,Td^i — 1} and Fi[n] = for n G {Td^i, ■ ■ .}. Let Gj denote the fraction 
of the first I message symbols that are in error, and Hn denote the time-rate of message symbols 
errors over the first N slots, i.e., 

/ 00 

Gi = 

i=\ n=l 
N-l 

= nY. Y 

”=0 {r-Td,i=n} 
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We note that lim/_^oo G/ = Pein)- H = limTv^oo Wat, then H = XPe{j) from [THl Section 
6 ]. For the batch transmission scheme, the set {i : = n} is empty, except for those n such 

that n = T[m] = Y^^=iT[m']. We also have that |{i : Td^i = T[m]}| = S[m — 1]. We note 
that since the channel is memoryless and the decoding of a message batch of size S[m — 1] is done 
based on the channel outputs received only in the transmission time T[m — 1], given S[m — 1] and 
T[m — 1], the events {Ei : = T[m\] are independent of any other error event of a message 

symbol transmitted in a period other than m — 1. As in the previous case we associate a reward 
R[m] with the renewal cycle, which is the total number of message symbols in error during 
the transmission period, starting with Q[m — 1] message symbols in the queue. The expected 
reward E — 1]] in the renewal cycle is 


r[m] = E 


Q[m - 1] 





[m] ,T[m] 1Q [m— 1] 

Pr{Ei\S[miT[m\] 

Q[m - 1] 





where Pr {Ei\S[m] = s,T[m] = r} is the probability of error of the symbol transmitted in the 
period using a block code with size |A^|^ and of codeword length r. We define the error 
cost at a decision epoch to be Cs(s, r) = r) P''" {^i\^ = s,T = r}, where at that decision 

epoch, C(s,t) is the set of s symbols which have been jointly encoded into a codeword of length 
T. We note that for 7 G T^, E^^E'j-jQT(Q) < 00 . Applying the Markov renewal reward theorem 
[331 Theorem D.16] we obtain that 


And therefore. 


lE„^Es,riQCs(S(Q),T(Q)) 

A ETr^E-y-jQT(Q} 


( 6 . 6 ) 


In the next section we discuss some approximations for Cs(s,t) which are used in further analysis 
of the tradeoff problem. 


6.2.1 The error cost Cs(s, r) 

We note that Cs(s,t) = Z)ieC(sr) pEi|S=s.T=T]' expected number of message sym¬ 

bols in error when a random block code is used to transmit s message symbols in r slots. In 
the following, the error cost Cs{s,t) is approximated using the Gallager random coding upper 
bound, since we are interested in an asymptotic characterization of the tradeoff curve as the error 
rate constraint Pe,c i 0. It is intuitive that policies 7 G T^j, for which Pei'y) < Pe,c, have 
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block lengths growing to infinity as Pe,c i 0. Therefore in the regime of Pe,c i 0. approxi¬ 
mating Cs{s,t) using the Gallager random coding upper bound is not unreasonable |23| Let 
P{s,t) = Pr{3i G {1, • • • , s} s.t mg^i / ms,i\S = s,T = r}, where nis^i and rhs^i are the trans¬ 
mitted and decoded symbols respectively. 

For a random block code of length r transmitting s symbols, we have from [221 Theorem 5.6.2] 
that the codeword error probability P{s,t) is 

< min (l, in |A4|)^ ^ ^-rEr{p,s,r,Q)^ ^ 

where Eo{p,Q) = - In (Lga: (dPvjxil/lx))^^ ^, p G [0,1] and Q is the chan¬ 

nel input distribution. The best upper bound is obtained by minimizing the above bound over the pa¬ 
rameter p G [0,1] and the distribution Q. We note that this can be done by maximising the exponent 
Pr(p, s, T, Q) with respect to p and Q. Depending on whether we do not choose/choose to optimise 
over p and/or Q there are four different exponents : (a) Er{p, s, r, Q) = {Eq{p, Q)—p^ In |Af |), (b) 
Er{p,s,T) = maxQ{Eo{p,Q) -pf ln|Af|), (c) Er{s,T,Q) = maxpgppj(£^o(p, Q) -pf ln|Wl|), 
and (d) Er{s, r) = maXpg[op] maxQ(T;o(p, Q) - pf In |Af |). 

If a symbol is in error then the decoded codeword is also in error, therefore Pr {Ei\S = s,T = r} < 
P{s, t). Using the union bound on P{s, r) and the above inequality we also obtain that 

S 

P{si t) < Pr {Ei\S = s,T = r} < sP{s, r). (6.8) 

i=l 

In the following, we approximate Cs(s,t) by assuming that all symbols are decoded incorrectly 
if the codeword is in error. Different upper bounds on the codeword error probability (from the 
different random coding exponents) can be used to obtain approximations for Cs(s, r). The following 
approximations are possible : Cs{s,t) = (a) smin(l, where both p and Q are 

fixed, (b) smin(l,e™“2(-TSo(p,Q)+P^in|A4|))^ ^ggre p is fixed, (c) se”“^e[o.ii(-ri?o(p,S)+psin|A4|)^ 
where Q is fixed, and (d) se™“'’S[o,i].s(-^®o(p,S)-i-psin|At|) approximation (a) is found 

to be analytically tractable as p and Q are fixed and do not depend on s and r. In (b) the optimal 
Q is a function of the fixed p and the following analysis applies, as it would be similar to that of (a) 
with Q being fixed at the optimal Q for the fixed p. We note that the minimizing p and Q are not 
known explicitly as a function of s and r even for simple channel models [221 Chap 5]. However, 
for R-model-A, we shall see that the exponential decay, of the minimum average error rate with 
the constraint on the average delay, can be shown to be governed by the best exponent (d), even 
with the following assumption. For R-model-B, we comment on how the asymptotic results can be 

'In m Gallager has shown that the random coding upper bound is tight for the probability of block error for an 
IID ensemble of codes, by showing that the lower bound on the ensemble error probability is at most O In 
away from the random coding upper bound for large block length r, for code rates less than the capacity of the 
channel Py\x- 


233 



extended to other approximations of Cs{s,t). We make the following assumption : 

Cl : The error cost or the expected number of symbols in error for a random block code of codeword 
length T transmitting s symbols is Cs(s, t) = s min (l, , where p and Q 

are fixed and are such that > Aln|Al|. 

The tradeoff problem that we consider is defined separately for R-model-A and R-model-B in the 
following sections. 


6.3 Asymptotic analysis for R-model-A 


6.3.1 Problem Statement 


We consider the tradeoff of average error rate with average delay for 7 G T^. The following analysis 
has been presented in [58J and [56J. The TRADEOFF problem is: 


minimize.yg5j, 

such that 


E^^Es,T\QCs{S{Q),Tm 

ae^^E7-|qT((5) 

^^■KjE-^^QTiQ) 


< Dc, 


(6.9) 


where we have used (6.5) and (6.6). If Sb = T^, then let P*{Dc) denote the optimal value of 
TRADEOFF. If Sb = Egj, then the optimal value of TRADEOFF is denoted by P* and if 
Sb = then the optimal value of TRADEOFF is denoted by P* We can also consider 

the equivalent problem EQT-TRADEOFF: 


minimize.yg5j^ 


such that 


E‘w^E-j-\Qh{Q,T{Q)) 

XET^^Ej-^gTiQ) 

E^^Es,r\QCsiS{Q),T{Q)) 

XETT^E-j-^gTiQ) 


< P. 


( 6 . 10 ) 


If Sb = Ts, then let D*{Pe^c) denote the optimal value of TRADEOFF. If Sb = Egj, then the 
optimal value of TRADEOFF is denoted by D*^{Pe^c), and if Sb = then the optimal value of 

TRADEOFF is denoted by D*j^ {Pe,c)- In the next section we discuss the asymptotic behaviour of 
P*j:{Dc) and P*{Dc) as Dc — 00. We note that as in Section 
an optimal policy for the above problem. 


4.2.1 


we can show that there exists 


6.3.2 Asymptotic analysis 

In the following lemma, we obtain uxin^P* 
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Lemma 6.3.1. We have that info^ ~ PncW' where 


r e-N4Eo{p,Q)-p\ln\M\) jf 0 < XNc < s', 

pnS^) = 1 , “ 7 , 

i f ~ s') + s'e ^<:^oip,Q)+ps in|A 4 | j ^ otherwise, 
where s' is the unique solution of the equation (1 +/?s'In |A 1 |) = gNcEo{p,Q)^ 


Proof. We first show that PnS^) ^ P^nSPc), '^Dc- We note that as we are considering policies 
7 £ the expected transmission time is always Nc and therefore ( 6 . 5 ) and (6.6) simplify to 

AX respectively. We consider the following optimization problem: 


1 


minimize 


such that 




5minfl,e-'^^®°(^’2)+p5ln|A^| 


XN, 

EttS > XNc + e, e > 0, 


( 6 . 11 ) 


where tt is any distribution for S with support on M+, irrespective of the policy 7. With e = 0 in 


(6.11) leading to the constraint E^^S > XNc, the optimal value of the minimization problem (6.11) 
is a lower bound to P*^^{Dc) for any finite Dc as the minimization is over all possible distributions 
of S and we impose only the constraint that the average service rate has to be greater than or 


equal to the arrival rate A. From Appendix 6.A we obtain that 


e,N, 


{Dc)> 


(,-N,{Eo(p,Q)-p\\n\M\) jf Q < 

{{XNc — s') + s'e-^c£;o(p,S)+ps'in|At|^ otherwise, 


( 6 . 12 ) 


where s' is the unique solution of the equation (1 +/?s'In |Af |) ^ for every 

finite Dc- Let Pnc{X) denote the RHS of ( 6 . 12 ). We note that Pn^{X) is equal to the value of the 
lower convex envelope of as a function of s, at s = XNc- 

Now we show that for any e > 0 , it is possible to construct a policy 7 G Ps,Nc such that Pe{'x) < 
PNAX) + e. Then we have that P*^^^{D{'j)) < PN^{X) + e and therefore PN^iX) = infu^ P*j,^^{Dc). 
In the first step of the construction we consider the optimization problem ( 6 . 11 ) but with an e > 0 . 


The optimal value of the problem ( 6 . 11 ) as a function of e is denoted by s{e). A feasible distribution 


for S that achieves s{e) + <5 is called ( 5 , e)-optimal for a <5 > 0 . In Appendix 6.A it is shown that 
there exists a distribution with finite support that is (< 5 , e)-optimal for any e > 0 and J > 0. 


Now we proceed to the second step of the construction. For an e > 0 we use any (J, e)-optimal 
distribution to construct a policy as follows. Let U[m] be a sequence of IID random variables 
distributed as the (( 5 , e)-optimal distribution. The policy 7 uses batch sizes 5 [m] = mm(( 5 [m — 
1 ], U[m]). We note that 7 is stationary. Since e > 0 , EC/ > XNc- Therefore, the EMC Q[m] under 
7 is positive recurrent. It can also be shown that the average delay is finite as U[m] has finite 
support. Therefore, 7 is admissible, i.e., 7 e r,. Evaluating the average error cost for 7 we have 
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that Pe( 7 ) < 


EcsiU,N,) 

XN, 


' ^epln\M\^-N^Eo{p.Q) + {pln\M\)\N^ ^g-N^Eo{p,Q) + {pln\M\){\N^+e) ^ if £ [ 0 ,s'-e], 

(^XNc -s' + + 1 ^, Otherwise. 


Let 6 = e. Therefore, Pe( 7 ) < PncW + C>(e). Hence, by an appropriate choice of e in ( ra , we 
can show that Ve > 0, there exists Dc = D{'y) such that P*j^^{Dc) < Pe{l) < Pn^W Hence, 
P^,(A)=infz,,P,>^(Pc). □ 


Remark 6.3.2. Since any policy 7 G is feasible for TRADEOFF for any Dc > D{^), we also 
have that PnS^) = iiif 7 er„ jv^ Peil)- We note that P*j^^{Dc) is a non-increasing function of Dc- 
Therefore, P/v^(A) = liuiD^^oo P^PncW is similar to c(A) in Chapter 4. 


We now present a lemma which formalizes our intuition that P/v^(A) approaches zero as the code¬ 
word length Nc approaches infinity. 

Lemma 6.3.3. If Aln|Af| < , then Pn^W = for large enough Nc, 

and therefore, liniAr^^oo PArc(A) = 0 . 


Proof. From Lemma [6.3.11 we have that 


I g-Ar,(Eo(p,S)-pAln|At|) jf q < < g/ 

PNc{X) = < . K 

i f(AA"c — s') -F s' otherwise, 

where s' solves the equation (1-F ps'ln |A4|) We note that if we show 

that, Aln|Af| < implies that XNc < s' for large enough Nc, then we have that PncW = 

^-Nc{Eo{p,Q)-p\\n\M\) tQ|. large enough Nc, which leads to both results of the lemma. We have 
that 


,-NcEo{p,Q) 


gp7ln|At| |i + ps'in|yw|} 


= 1 or 


yos'In |Af I-F In (1-F s'pIn |A4|) = NcEo{p,Q). 


(6.13) 


We note that the LHS of (6.13) is an increasing continuous function of s' with LHS = 0 at s' = 0. 


Thus for any Nc, there is a unique s' which solves the equation (6.13). Also as Nc increases this 


unique solution s'{Nc), as a function of Nc, also increases, and as Nc t 00, s'(iVc) t 00. We 


multiply both sides of ( 6.13 ) by A, divide by s'{Nc) and Eq{p, Q) to yield 

p , A ln(l+ s'(iVc)pln|M|) XNc 


Aln|M| 


-F 


Eo{p, Q) Eo{p, Q) 


s'iNc) 


s'{Nc 


(6.14) 


Since we have assumed that Ain |A1| < 5 = 1 — Ain |-A4| > 0. For sufficiently large 

Nc, s'{Nc) can be made sufficiently large, so that '”( 1 +^ | Eo{p,Q) jmpijgs that 
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< 1 - Hence for large Nc, < 1 and therefore Pat,(A) 

Therefore, Pn^W i 0 as t oo if Aln|A^| < . 


^-N,iEoip,Q)-p\ln\M\) ^ 
□ 


We now consider the asymptotic behaviour of P*^{Dc), as Dc t oo- We first show that if A In \M\ < 

^ then P*j-{Dc) i 0 as Pc t oo- This result is obtained by showing that for a sequence of 
policies /ifc G T^j, for which D{fik) f oo, Pe{l^k) i 0 . Then P*j{D{^k)) | 0. Furthermore, 
since t oo, P*j{Dc) 4- 0 for any sequence Dc t co. The sequence of policies Hk G Tgj is 
parametrized by the codeword length Nc^k< where A^c.fc is an unbounded increasing sequence. A 
fixed rate of transmission r is chosen such that Aln|A/l| < rln|Af| < . Let Rk = rNc^k- 

The policy chooses 5[m] = min(Q[m — l],Rk)- We note that since r > A, it can be shown 
that fj.k £ V/r. 

Proposition 6.3.4. lfAln|Af| <rln|Wl| < for the sequence of policies /Xfc defined above, 

we have that linifc^oo = r-\+r\ {Eo{p,Q) - pr\n\M\), with P(/Xfc) t oo- Therefore 

Pe{pk) i 0. 


[5e“^c,feSo(p,Q)+pSin|AX|j 

Proof. For /x^, we note that Pe{pk) < PPpP - — -i 


Vc,fc 


. As S < rNc^k we have that 


Pe{Pk) ^ 


-Nc,kEo(p,Q)+pSln\M\ 
-N.k 


< _ ^ ^Eo{p,Q)-prln\M\ 


(6.15) 

(6.16) 


From Appendix 6 .D we have that 


'^PkQ — 


<7^ ^ XNc,k , rXNc,k 

r — X 


2{r-X) 


+ 


+ 


We note that in our case, there is an extra time average holding cost of due to the 

customers waiting during a transmission period of A^c,fc slots. Therefore, 


D{pk) < 


r — X 


a 


2 A(r - A) 2 


~ 7. P P^c,k ( 1 + 


rX 
r — X J ’ 


r — X + rX \ 


( P(^fc) - 


a 


1 


2 A(r - A) 2 






(6.17) 


Since D{fik) is at least Nc^k we have that D{iJ,k) t oo as A^c.fc t oo. 


Substituting the lower bound on Nc^k froni (6.17) in (6.16), we have that 

Pe{pk) < - (e^o{p,Q)-prln\M\yP^^i^^^'^^~^^^)^^ 
Hence limfc^oo = r-x+rX (-®o(p, Q) - prln|Al|), and Pe{pk) i 0- 
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□ 


We now show that any sequence of policies 7^ G ^sj, for which Pei'yk) —t 0 , has Nc{'yk) —t 00. 
Lemma 6.3.5. For any sequence of policies 'jk G Nd'yk) —t 00 if Pe{lk) — t 0 . 


Proof. Assume that there exists a sequence of policies 7 ^ G Tgj for which Pei'Jk) 0. but 
Ncilk) 7 ^ oo- Therefore limsup^^go Arc( 7 fc) = Nc^i < 00 . From Lemma 6.3.1 we have that 
Pehk) > TAr,,fc(A),VA:. If Nc^i < 00 we have that liuik^oo Pehk) > Pn,,iW > 0, which contra¬ 
dicts the assumption that Pe{lk) i 0. Therefore Nc{-jk) should necessarily grow to infinity for the 
sequence of policies 7 ^. □ 


From the above result and since P*j{Dc) i 0 as He t oo. we know that to obtain an asymptotic 
characterization of P*^{Dc) as Dc t 00 , we need only consider sequences of policies 7^ G Tgj with 
Nci'Jk) t 00. Now for any sequence 7^ G T^j, with N^jk) —t 00, we present an upper bound on 
the decay rate liniD^^oo 


HPlfiD,)) 
Da 


Proposition 6.3.6. If Aln|Al| < , the exponential decay rate of P*^(Hc) has the following 

upper bound: 

Dc —^00 c 

for a fixed p and Q. The best upper bound on the decay rate is obtained by fixing p and Q to be 
p* and Q* respectively, where 


(p*, Q*) = argmax^ (Eo(p, Q) - / 9 Aln|Al|). 

P6[0,1],Q 


Proof From Proposition 


6.3.4 


we 


have that P*^{Dc) i 0 as He t oo. From Lemma 


6.3.5 


we need 


only consider any sequence of 7 ^ G T^j such that Nc^k = Nc{lk) 
that Pei'Jk) > -P/Vcfc(A). Then from Lemma 


6.3.1 


6.3.3 


00 . We have from Lemma 
for large enough k, as Aln|Al| < 


we have that Pe{lk) > e pAin|At|) Prom (6.5), for a fixed codeword length A^c,fc we 

have that D{'yk) = + ^‘'’2 ^ ■ For any 7 ^, = AA^c.fc as Q > S. Therefore 

H( 7 fc) > ^ ^ or i(H( 7 fc) + A) > N,^k- Hence Pe( 7 fc) > |At|) 

or limfc-».oo — i(-^o(P)Q) — pAln|Al|). We note that the above upper bound on the 


exponential decay rate holds for any sequence of policies jk- For ( |6.9D , we have by definition that for 
He > 1 and e > 0, there exists a 7 such that D{'y) < Dc and Pd'y) < P*j^{Dc)+e. For any sequence 
7 fc, choose e from the sequence . Then, we have that limDa^oo < 


|(Ho(/ 0 , Q) — /oAln|Al|). The best upper bound on decay rate is then obtained by choosing the 


fixed p and Q to be p* and Q* respectively. 


□ 
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Now we consider the decay rate, lim£)^'|-oo —Unlike the class of policies considered above, 
where the transmission duration was always fixed to be a parameter Nc, for 7 G we have that at 
each decision epoch m, the transmitter can choose both the batch size S[m] and the transmission 
duration T[m]. We obtain a lower bound to the exponential decay rate of P*{Dc) as t 00 , by 
obtaining the exponential decay rate limfc_^oo ^ sequence of exhaustive service (EXH) 

policies efc, with D{ek) t oo- Consider an EXH policy e. The policy e chooses 5[m] = Q[m — 1] and 
T[m] such that the codeword error probability is at most Intuition suggests that the codeword 
length T[m] has to be at least a minimum value, say r(S'[m]), which is a function of the batch size 
S[m], to guarantee that the codeword error probability P{S[m],T[m]) is at most Pe,6 foi' every m. 
The message alphabet size is |Af |, therefore the alphabet size of the batch of s message symbols 
is The rate in nats '\s R = Consider a randomly generated codebook, in which each 

codeword symbol is chosen independently according to the distribution Q on the input alphabet 
X. The receiver is assumed to do maximum likelihood decoding of the joint message. From [221 
Theorem 5.6.2], we have that the average probability of codeword error is bounded above as: 


P{s,t) < 

or lnP(s,r) < /?s In |A^| - t£'o(p, Q), 


where p G [0,1], Eq{p,Q) = — In f f Q{x)PY\x{y\x)^^'^^'^ dx dy. To guarantee the 


Jy&y VJ x&X 

average error rate requirement, we constrain P{ 


p-T{-pR+Eo(p,Q)} ^ p 


then P{s,t) < Pg.r- For every s, if r is chosen as a function r(s) to satisfy the above inequality, 
we have that 


-lnPe,r pslnjWlI ^ 

Eo{p,Q)^ Eoip,Q) - 

Thus t{s) has to be chosen as the smallest integer greater than or equal to as+b, where a = 
and b = ~Eq{pQ) ■ Therefore t(s) = [as + 6] , s > 0, so that P(s, r) < Pe,r- 

We assume that e chooses T[m] = r(S'[m]) if 5[m] > 0 and T[m] = 1 if S'jm] = 0. Let the 
stationary distribution corresponding to policy e be tt. Then for e, T’e(e) is less than or equal to 
Since EttS = AE^rT we have that Pe{e) < Pe,b- For a given Pe,fe let the average delay for 
e be denoted by DEXH{Pe,b)- We have the following upper bound Du{Pe,b) on the average delay 

DEXH{Pe,b)- 

Proposition 6.3.7. DEXH{Pe,b) < Du{Pe,b), where 


Du{Pe,b) 


6+lA / acr^ 3(5 + l)A aa'^ X\ 

b ) V2(l -aA) 2(1 - aA) 2(1 - a^X^) ~ 2 ) ' 
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The proof is given in Appendix 6.B[ We now compare the above upper bound with the upper bound 
obtained by Musy and Telatar |40]. We note that the model considered by Musy and Telatar in 
[40j is a continuous time model with Poisson arrivals. In [40j, if a batch size s > 0 is used, then 
the service time \s D + ks. For simplicity, assume that D and k are integers. Let 6 be such that 
b6 = D and a6 = k, where a and b are integers. Then if we assume that each slot in our model is 
of 6 duration, then the time taken for service of s customers in our model and the model in [40J is 
the same. Let the Poisson arrival rate of customers in Musy's model be Am- Now assume that the 
arrival process in our model is Bernoulli with an arrival probability (rate) of A = Am<5 in each slot. 
We note that if a and b are integers, then r(s) = as + b for every s > 0. Then the upper bound in 


Proposition 6.3.7 simplifies to 


aa 


+ 


3bX 


+ 


aa 


A 


2(1-aA) 2(1-aA) 2(1 - o2A2) 


(6.18) 


We note that cr^ = A — A^ for a Bernoulli arrival process. Substituting A = Xm^, a = j and b = ^ 
and taking the limit as (5 I 0 (along a sequence such that a and b are integers) we obtain that the 
average queue length in the limit is 


3DXr 


+ 


k{2 + Xmk) 


2{1-X^k) 2(1-AmA:) 

We note that this is the same as that obtained by Musy in Section 2.3.2 of his thesis 


(6.19) 


The above upper bound leads to the following characterization of the exponential decay rate of the 
average error rate. 


Proposition 6.3.8. The exponential decay rate : liuiD^^oo> |(-Eo(/ 0 *j Q*)—/c^Aln |A4|) 


Proof. Let Pe,uiDc) = inf {p G [0,1] : Du{p) < Dc}. We note that if Du{p) < Dc, then the 
e policy corresponding to p has D{e) < Dc and therefore P*{Dc) < T’e(e) < p. Therefore, 
P*{Dc) < Pe,u{Dc). We have that Pe^u{Dc) is such that 

/' bu + l \ /' 3{bu + l)X ao-2 x\ _ 

\ buX y V 2(1 - aA) 2(1 - oA) 2(1 - a2A2) 2 ) ~ 

where bu = — Eo{pQ) obtain the following quadratic equation in bu- 

(bu + 1) {{bu + l)ci + C 2 ) = buXDc, 


240 

















where ci 


3A 

2(l-aA) 


and C 2 


atj^ ( 2 +aX) 
2{l-a^A2) 


i- Or 


-lnPe,»(-Dc) 

Eo{p, Q) 



2ci 


C2 + \/{XDc - 2 ci - C2Y - 4 ci(ci + C2) , 


since we want the largest exponent. Let Dc,A: be a sequence such that Dc,fc t 00. Then, we have a 
sequence pk = inf {p : Du{p) < T*c,fc}- As above, we obtain 


-Inpfc 

Q) 


1 




C2 + ADc,fc 


2ci+C2\^ 4ci(ci + C2) 


Therefore, we have that j,_^oo “zJ^ ~ _ ^(^Eq{p, Q) — pXln |Af |), after substituting 

for a in ci. Hence, we have that lim/j^^oo > |(So(/?*, Q*) — /?*Aln|A4|), by fixing p 

and Q to be p* and Q* respectively. □ 


We note that we do not have any upper bound on the exponential decay rate 
over the class of policies T^. 


6.3.3 Integer valued queue evolution 


We note that for R-model-A, the queue length process evolves on M+. Suppose we consider the 
case, where the arrival process A[m] G {0,1,..., Amax G ^+}, W £ and 5[m] G Z+. Then the 
queue length process {Q[m],m > 0) would evolve on Z+. Let us denote this model as l-model-A. 
Intuitively, P*{Dc), P2j{Dc), or P^j^^^Dc) (or T>*(Pe,c), Dj{Pe^c), or D*^^{Pe,c)) for R-model-A 
(with Pr {A[m] = a} > 0 only for a G {0,1,..., ^max}) would be a lower bound to the same 
performance measures for l-model-A, as the set of feasible policies for R-model-A would always be a 


superset of the set of feasible policies for l-model-A for the optimization problem (6.9) (or (6.10)), 
from which P*{Dc), Plf{Dc), or P*j^^{Dc) (or D*{Pe,c), D*^{Pe,c), or Dy^{Pe^c)) is obtained as 
the optimal value. 


For l-model-A, we note that P*j{Dc) i 0 as He t 00 , since the sequence of policies pk can be 
restricted to use only non-negative integer valued Rk = rNc^k by an appropriate choice of the 
sequence A^c,fc and a rational number r > A. Furthermore, in this case, the following tighter bound 
on the exponential decay rate can be obtained for the sequence pk- 


Proposition 6.3.9. If Aln|Af| < rln|A 4 | < for the sequence of policies pk defined 

above, we have that lirrifc^oo ~ = i {Eo{p, Q) — pr\-n.\M\), with D{pk) t 00 . Therefore 

Pe{pk) i 0. 
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[5e“^c,fcBo(p.2)+pSin|Al|l 

Proof. For we note that Pe{^ik) < —-xiv- - 


Pe{Pk) < 


. As 5 < rA^cfc we have that 

g-iVc,fe£;o(p,S)+p51n|Atr 


< 


^ /'pEo(p,S)-pr-ln|At| 


( 6 . 20 ) 

( 6 . 21 ) 


From Denteneer et al. [I^ equation (9) and the upper bound in (12)] we have that 

+ ^min(AiVc,fc>?^^c,fc - !)■ 

For k large enough, as oo and A < r, we have that 

^2 


iEttu, Q ^ 


a 


“ 2(r - A) 


+ fc. 


We note that in our case, there is an extra time average holding cost of due to the 

customers waiting during a transmission period of Nc^k slots. Therefore an upper bound on the 
time average queue length is 

3AA^c,fc A 


cr 


2(r - A) 


+ 


Therefore 


^^^"^-2A(r-A)+~"2’ 


- D{^ik) - 


a 


2A(r - A) 2 


+ “ 1 ^ Nc.k- 


( 6 . 22 ) 


Since D{^k) is at least N^^k we have that D{^k) t oo as A^c,fc t oo. Substituting the lower bound 
on Nc^k from (6.22) in (6.21), we have that 

PeiPk) < J (^e^o{p,Q)-pr\n\M\^ 

Flence limfc^oo ~ ^ ^ (-^o(/0, Q) - prln\M\), and Pe{pk) I 0. □ 




2A(r-A)"^2 


Remark 6.3.10. We note that Proposition |6.3.9 provides a lower bound on the exponential decay 
rate of linifc^oo since for any D^^k t oo, we have a subsequence of pk, such that 

D{pk) < Dc^k and therefore P*f{Dc^k) < PeiPk)- We note that the capacity (C) of the discrete 
memoryless channel is given by ^ lp=o- Let us consider the case when C — 6 < Ain |A1| < C, 

where (5 is a small positive constant. In the following, we show that the lower bound to the expo¬ 
nential decay rate of P* j{Dc) achieved by the sequence of policies pk approximately matches with 
the upper bound on the exponential decay rate of P*j{Dc), ^ {Eo{p*, Q*) — pAln|Al|), obtained 
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in Proposition |6.3.6 We note that if C — <5 < Aln|Af| < C, for small positive S, then any p satis¬ 


fying Ain \M\ < 


go(p.Q*) 


is approximately zero. Furthermore, p* is such that Aln|Wl| < 


go(p*.g*) 


and is therefore approximately zero. Then, from (6.20), since E, 


,p*S\n\M\ _ „p*AAr,,;,ln|At| 


have that Peipk) ~ p*Ain|At|^ which yields an approximate lower bound on the 

i {Eo{p*, Q*) — p*Aln |A/1|), which matches with the 




we 


exponential decay rate lim 


■k^oo 


-lnPe(pfc) 

D(pk) 


upper bound in Lemma 6.3.6 


Recall that Pn^{X) = inf^)^ P*j^^{Dc) for both R-model-A and l-model-A, where we have restricted 
to the set of policies for both models. We have that Proposition 6.3.6 also holds for l-model- 

-.6 holds for l-model-A as: (a) P*j(iAc) | 0 as Dc t from 
and (b) Pei'Xk) > which holds since Pn^W for R-model-A is a lower 


A. The proof of Proposition 
Proposition 


6.3.9 


bound to PncW defined for l-model-A. 

We also note that the upper bound Du{Pe,c) holds for l-model-A under a EXH policy which serves 


only integer number of message symbols. Hence Proposition 6.3.7 and therefore Proposition 6.3.8 
also holds for l-model-A. 


Remark 6.3.11. In the next section, we present an asymptotic analysis for R-model-B. We recall 
that for R-model-B, the codeword length Nc is fixed. The asymptotic analysis of R-model-B is 
significant, since the tradeoff problem for R-model-B is a subproblem for R-model-A with the 
restriction to policies in T^j. In fact, we study how i9|^^(Pe,c) behaves as Pe,c i PncW- 


6.4 Asymptotic analysis for R-model-B 


6.4.1 Problem Statement 


We state the tradeoff problem for R-model-B so that it is similar to the definition of the problem 
TRADEOFF in Chapter 4. The TRADEOFF problem for R-model-B is 


mimmize..),grs 
such that 


E.„^Eh{Q,Nc) 

AiW 

E^^E5|QC,(5(Q),iV,) 

XN, 


< P 

— ^ e,ci 


where h{q,Nc) = qNc -F as defined before, and is a constraint on the average error 


rate. As in Section 4.2.1 we can show that there exists a set of Pp r such that there exists a 


stationary deterministic optimal policy for Pg^c £ This stationary deterministic policy is optimal 
for an unconstrained MDP with single stage cost h{q,Nc) + ^Cs{s,Nc), where > 0 is a 
Lagrange multiplier. 

However, we consider the above problem only for a subset Ta c r„ which is the set of monotone 
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admissible policies. We redefine the set of admissible policies for R-model-B as follows. A policy 

7 e Ta if: 


RGl : 7 G r„ 

RG2 : it induces an aperiodic, irreducible Harris Markov chain Q[m], 

RG3 : the average service rate at a queue length q, E5(g) is non-decreasing in q. 


We note the above properties are similar to those defined in Section 4.4 of Chapter 4. 


We note that the function Cs{s,Nc) is not convex. Consider a Pe^c G and the corresponding 
unconstrained MDP with Lagrange multiplier Existing proofs of the monotonicity property of 

the optimal policy for the unconstrained MDP, such as those in Goyal et al. |25j, require that the 
function Cs{s,Nc) be convex. We are not able to prove that the batch size is a monotonically non¬ 
decreasing function of the queue length for the optimal policy even if Pe,c G unlike in Chapter 
4. However, we observe that the optimal policy prescribes a batch size which is monotonically 
increasing in the queue length in numerical solutions of the MDP. This is the only motivation for 
assuming RG3. 


The TRADEOFF problem for R-model-B is to obtain L)^^(Pe,c) which is the optimal value of 


minimize.ygra 

such that 


E^_^Eh{Q,Nc) 

AiVe 

^^51Q Cs ( 5* ( Q ) , A^C ) 

Aiw 


<Pe. 


We note that for every Pe,c such that the above problem is feasible, for every e > 0, by definition 
there is a feasible admissible policy 7 such that D{'y) < D'^^{P(,^c) + We call such an admissible 
policy e-optimal for Pe,c- 

For R-model-B, suppose that Smax > Chapter 4, we define c(s) : [0,5max] —11^+ 

as the lower convex envelope of {{s,Cs{s,Nc)),s G [0,S'maa;]}- Then there exists a s' < Smax, 
which satisfies the following equation: 


[Smax + pin \M\s'{Smax - s')] . 


We note that the tangent drawn from {Smax,Cs{Smax,Nc) = Smax) touches the Cs{s,Nc) curve 
at {s',Cs{s', Nc)). We also note that the definition of s' is similar to that for R-model-A, except 
that for R-model-B, the slope of Cs{s,Nc) at s = s' is not one. Furthermore s' for R-model-B is 


always greater than or equal to s' for R-model-A. Figure 6^ shows an example. We note that the 
lower convex envelope c{s) = Cs{s,Nc), for s G [0,s'] and for s G {s', Smax), c{s) is the tangent 
line segment drawn from {s',Cs{s', N^)) to {Smax, Smax)- 
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Figure 6.2: Illustration of the error cost function Cs{s,Nc) and the lower convex envelope c(s). The lower 
convex envelope coincides with Cs{s,Nc) for all s G [0, s']. 


For R-model-B, the infimum of all achievable average probabilities of error, inf.ygra ^e(7), is again 
denoted as P/v^(A). Similar to the proof of Lemma 


6.3.1 


we can show that P, 


N, 


W = S:- the 


following, we obtain an asymptotic lower bound to {Pe,c) 3S Pe,c i Pn^W- 


6.4.2 Asymptotic analysis 

We assume that the following properties hold for ^[Ij. 


RAl : Pr{A[l]-Smax>Sa}>ea, 

RA2 : Pr {^[1] < = e'^j > 0, for some 0 < < s'. 


We note that RAl is the same as that defined in Section 4.4 of Chapter 4. We also note that the 


property RA2, is similar to the property A2 assumed in Section [4.1.2] of Chapter 4, in that there is 
a positive probability of the number of arrivals being in an interval including zero. 

We note that the service cost function c(s), as defined above, satisfies the following properties. 


which are similar to the properties RCl and RC2 defined in Section 4.4 of Chapter 4. 


RCl : c(0) = 0, and, 

RC2 : c{s) is strictly convex for s G [0,s') and linear for s G [s',5max]- 

The asymptotic behaviour of D'^^{P(,^c) as Pe,c i Pn^W is different depending on whether X < 
or A > We characterize the asymptotic behaviour in both of these cases in the following. 

Lemma 6.4.1. For A < ^, and for any sequence of admissible policies jk with Pe{lk) ~ Pnc{X) = 
Vfc 4- 0 we have that 

" (t^) ■ 
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Proof. For an admissible policy 7 , define C{'j) = c{S{Q)). We note that as Cs{s,Nc) > c{s), 


Pe(7.) = > 


0(7*) 


. Then we have that there exists a sequence Uk such that Uk = 


AiVc — XNc 

C{lk) — c(A) < \Nc {Pe{jk) — Pnc{^)) = ^NcVk i 0 . Let us consider a particular policy 7 in the 
sequence 7 ^ with Uk = U and = vr. To obtain a lower bound on Q{'y) we proceed, as in the 


proof of Proposition 4.4.2 with = sup{g : ES{q) < XNc + ejj} where ejj will be chosen in the 
following. We follow all the steps in the proof of Proposition 4.4.2[ with qd as above and ejj in 
place of ey, up to the step where < 5 ( 7 ) is bounded below by where the k 2 is the largest integer 


satisfying the inequality (4.27). That is, we have that ( 5 ( 7 ) > where k 2 is the largest integer, 


such that 


1 + 1 ^ 

OCa 


k2 


< 


1 + 2A’ 


where Dt = P (ES{q) — XNc)d7r{q). We note that in Proposition 


4.4.2 


a lower bound on k 2 
the function 


4.4.2 


was obtained via the upper bound | on Dt- We note that, unlike in Proposition 
c(s) is not strictly convex for s G [0,5maa:]- So we derive an upper bound on Dt in a slightly 
different way. Define q\ = sup{g : ES{q) < XNc}. Let l{s) be the tangent line to the curve c(s) 
at (A, c(A)). We have that E^r {c(5((5)) — 1{S{Q))} = U. That is, 


/ 


E{c{S{q))-l{S{q))}dn{q) = U. 


As c(s) > l{s) we have that 


rqx 


10 


E{ciSiq))-liS{q))}dniq)<U. 


As c(s) is convex and l(s) is linear we have that 

rqx 

/ {c(ES((?))-/(E%))}d7r(g)<[/. 

Jo 

We note that for A < there exists an ai > 0 such that c{s) — l{s) > ai{s — XNc)"^ for s < XNc. 
Furthermore as for q < q\,ES{q) < XNc, we have that 

r {ES{q) - XNcf d7r{q) < 

Jo 


which can be written as 


rqx 


{ES{q) - XNcf d7r{q) + 


'qx 


U 

0d7T{q) < —. 

01 
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By Jensen’s inequality we then have that 


rqx 


roo 

{KS{q) — XNc) dTT{q) + / 0d7r{q) 

dqx 


ai 


Hence we obtain that 


r {ES{q) - XNc) d7r{q) >-J—. 

lo V ai 


(6.23) 


Now we note that for an admissible policy 7 , 

/ iES{q) — XNc) dTr{q) = 0 , or, 

Jo 

rqx rqd roo 

/ {KS{q) — XNc) d7r{q) + / {KS{q) — XNc) dTr{q) + / {KS{q) — XNc) dvr(g) = 0. 

Jo Jgx Jqd 

For q > q\, ES{q) > XNc, so that {ES{q) — XNc)dTr{q) > 0, which implies that 
rqx roo 

/ (E5(q') - AA^c) + / {ES{q) - XNc) dTT{q) < 0, 

Jo Jqd 

foo rqx 

/ {ES{q) - XNc) dTr{q) < - {ES{q) - XNc) diriq) < 

J Qd Jo 


from (6.23). Therefore we obtain that Dt < We choose eu = and proceed as in 


the proof of Proposition 


4.4.2 


to obtain that < 5 ( 7 ) > ^ 'i (|) “ ^ ■ Therefore, for the 




□ 


sequence of policies 7 ^, we have that Qi'jk) = ( 7 ^) ~ ^ — ^^oVk- 

^ ' then for all A < the 


Remark 6.4.2. We note that if Smnr is such that Smnr < 


above asymptotic lower bound would hold. 

Proposition 6.4.3. For A < as Pe,c i PnSJ^)’ have that 

D*{Pc^c) = Xl 


xjPe,c - Pn, (A) ) 


Proof. Consider the sequence of policies 7 ^ which are e-optimal for TRADEOFF for a sequence pk 


of Pe,c such that pk i PncW fo'' some e > 0. Since Pei'yl) — Pk: have that from Lemma 6.4.1 

that Qi'jl) = fi ■ Since D*(pk) > [QilD + ^Nc{Nc - 1)] - e, we obtain that 

D*{pk) = n( \ ). □ 

\ x/Pk-PNcP) J 

Remark 6.4.4. We note that an asymptotic upper bound, which is tight upto a logarithmic factor. 


can be obtained for the above case, using a sequence of policies as in Lemma 4.4.3 We note that 
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for this sequence of policies, it is possible to choose batch sizes s such that Cg {s,N,) = c{s). 


In the following, we obtain an asymptotic lower bound to iPe,c) as Pe,c i PncW fo'' > jr- 


This lower bound is obtained by extending Lemma 4.3.8 to the case when the state space of the 


Markov chain is the set of non-negative real numbers. As in Lemma [4.3.5| we first obtain a lower 
bound on the queue length as a function of the stationary probability of the queue length being in 
a certain set. Then we relate the stationary probability of the queue length being in the above set 
to the average error rate. 

Let Qs' = inf {q : E5(g) > s' — e}, where e is a small positive constant, chosen such that Aq < s'—e, 
where is as in RA2. We note that for an admissible policy 7 , E 7 rES'(Q) = XN^, and therefore 
there would exist finite q, for which E5(g) > XNc- Thus, qg/ is finite. The following lemma shows 
that for a queue length q > qg', there is a positive minimum probability of serving at least a certain 
number of customers. 


Lemma 6.4.5. For an admissible policy 7 , with g<j' defined as above, for Aa as in RA2, we have 
that inf„>g , Pr{S(q) > Aq} > > 0, where 6g = . 

^ L \ / j Omax —AAq, 


The proof of this lemma is similar to that of Lemma 5.7.1 and is presented in Appendix | 6 .C 


Lemma 6.4.6. For A > and for any sequence of admissible policies 7 ^ with Pe{'yk) — Pn^{X) = 
14 i 0, we have that 


Qi'fk) = X} 




Proof. For an admissible policy 7 , define C{'y) = We note that as Cg{s,Nc) > c{s), 

we have that there exists a sequence Uk such that Uk = C{jk) — c(A) < XNc {Pd'jk) — -P/Vc(A)) = 
XNcVk i 0. We consider a particular policy 7 in the sequence 7 ^ with Uk = U and TT.y^ = tt. As in 
the proof of Lemma 5.7.2[ since 7 is admissible, we have that if mi is the largest integer such that 


7r[0,gs/) -F7r[0,g^/) 


1 

1 + - 
P 


mi 


- 1 


= TT [0,^^/) 


IN mi ^ 

1 + - 1 < - 

P 


2 ’ 


(6.24) 


then Q('y) > . Let us define the line l(s) as the line passing through (s', c(s')) and (Smax-, ciSmax))- 

For the policy 7 we have that 

¥.{c(S(q))-l(S(q))]d7:{q) = U. 

As c(s) is convex and l(s) is linear we have that 

{c(ES(q)) - l(ES{q))}d7r(q) < U. 
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Also as c(s) > l(s), 




{c(ES(g)) - l(ES(q))}d7r(q) < U. 


We note that for q < qgi we have that ES{q) < s' — e and there exists ai > 0 such that 
c{ES{q)) — l{ES{q)) > ai{ES{q) — s'^. Hence we obtain that 


ris' 


{ES{q)-s'f <-, 

10 


and as q < qg', {ES{q) — s'f > e^. Therefore we obtain that 


r e^dTTiq) < 
lo 0,1 


And hence 7 r[ 0 , 


From (6.24), if m 2 is the largest integer such that 


m 2 


1 \ * oie‘ 

' + ^ W°'’ 




then m 2 < mi. We note that m 2 is at least 


(w) 


Since Q{'y) > , we obtain that 


aie 


Q(7) > - j - 1 j . 

So for the sequence of policies 7 c with Uh 1 0 we have that Q(jk) — ^ (iog (tt)) ~ ^ (f^))' 


□ 


Proposition 6.4.7. For A > as Pe,c i PncW^ we have that 


D*{Pe,c) = log 


Pe,c - PNcW 


The proof of the above result is similar to that of Proposition 6.4.3 


Remark 6.4.8. An asymptotic upper bound to iAJ^^(Pe,c). as Pe,c i Pn^W can be obtained from 
Lemma 4.3.10 We note that in this case, s/ = s' and Su = Smax- Then, as in Lemma 4.3.10 we 
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can show that there exists a sequence of admissible policies 7 ^, for which D*{Pe,c) = O ^log 


and Pe{lk) — PncW = f4 | 0. We note that Peijk) can be computed as in Lemma 4.3.10 

C-sis , Nc) — c(s ) and Cs{Smaxj -^c) — c(5*max)- 


Since 


Remark 6.4.9. We can set up a queueing model l-model-B, which is similar to R-model-B, ex¬ 
cept that qo G Z+, A[m] E {0,1,..., E Z+}, and S[m] E {0, 1, ..., S'^ax} , m > 1. 
Therefore, for l-model-B the queue length evolution is on Z+. We note that queueing model 
l-model-B is analogous to l-model-A. Asymptotic lower bounds to Zl^^(Pe,c) can be obtained as 
in Section 4.3.3 of Chapter 4, with c(s) defined as the piecewise linear lower convex envelope of 
{{s,Cs{s,Nc)),s E {0,. .., S'max}}- Asymptotic upper bounds to D*^^{Pe^c) can be obtained as in 
Remark [6.4.81 

Remark 6.4.10. For R-model-B, we note that asymptotic lower bounds can be derived for other 
approximations for Cs{s,Nc), if the lower convex envelopes for such approximations have the same 
form as c(s) above. We note that the asymptotic nature of the bounds only depended on: (a) c(s) 
being strictly convex in [0,s'], and (b) c{s) being linear in (s',5max]- In Appendix 6.E we consider 
some examples for the approximation Cs{s,Nc) instead of Cs{s,Nc) for the error cost, where 


Cs (s, Nc) 


min g^-^cEo(p,Q}+psln\M\ 

pe[0,i] 


and illustrate that the lower convex envelope c{s) of Cs{s, Nc) as a function of s has a similar form 
as c{s). 

We note that instead of using Gallager's random coding upper bound we could also use Polyanskiy's 
normal approximation for the codeword error probability to derive an approximation for Cs{s,Nc). 
From [49], if 0 < s < . then we have that the codeword error probability P^^h satisfies the 

following approximation: 


Nc 


Q-HPe,b) y 
1 _ glog2 I At I ) (J2 ’ 

N^C / 


where Q is the Gaussian Q function, C is the channel capacity (in bits/channel use), and V is the 
channel dispersion. Then, we have that 


P 


e,b 


NcC^ slog2|M| 

y V NcC 


and the approximation 


Csis,Nc) = 


NcC^ f^_slog^\M\ 


V 


NcC 


We note that if s > |^| . then we have the approximation Cs{s,Nc) = s. 
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It can be shown that if 
Proposition 


; log2 l-^l 


Q 


6.4.3 


-jq - < C, then Cs{s,Nc) is strictly convex in s. Then, as in 

we have that for any A < as Pe,c i ws have that D*{Pe^c) = 


\/pI^c-PnJP) 


6.5 Conclusions 


We have shown that for R-model-A (from Proposition 6.3.6) as well as l-model-A (the discussion in 


Section 6.3.3), the exponential decay of average error rate with average delay is at most two-thirds 
of the Gallager random coding exponent, when we restrict attention to the set Tgj of policies. We 
also note that for l-model-A, for information arrival rate Aln|Af| approaching the capacity of the 
channel, a sequence of policies e r.j, that uses a fixed service rate, approximately achieves the 
best exponential decay rate of two-thirds of the Gallager random coding exponent. We note that 
the I factor arises because of the fundamental limitation of block codes; message symbols arriving 
in a transmission period of duration Nc have an average waiting period of ^ until the succeeding 
transmission period and have to wait for at least an additional Nc slots before leaving the queue. 
Therefore, the average delay is at least |A^c- So naturally the question arises whether streaming 
codes have a better exponential decay rate. In [57|, we show that a sequence of fixed rate randomly 
time varying streaming codes, with increasing constraint lengths achieve an exponential decay rate 
of average error rate with average delay which is equal to the Gallager random coding exponent. 


We then considered the exponential decay rate for a sequence of exhaustive service policies, which 
ensure a constant block error probability per transmission by varying the codeword length. We 
obtain that for the above sequence of policies, the exponential decay rate is at least two-thirds of 
the Gallager random coding exponent. The performance of such policies has been studied in detail 
in [Sj, [54], |55j, [59J, and [60J. We note that the above exponential decay rate provides a lower 
bound to the exponential decay rate achievable by a sequence of policies in T^. However, we do 
not have an upper bound on the exponential decay rate over any sequence of policies in T^. 


The analysis of R-model-B illustrates the application of the lower bounding technique in Chapter 4 
to cases where the service cost function is not convex. We observe that the asymptotic lower bound 
depends on the nature of the lower convex envelope of the service cost function at A. Since the lower 
convex envelope c{s) is convex, the analysis and observations from Chapter 4 apply. We note that 
the form of c(s) considered in this chapter is not strictly convex or piecewise linear, but piecewise 
convex. The asymptotic upper bounds show that a sequence of admissible policies using only the 
service rates s for which the service cost Cs{s,Nc) = c{s) is order optimal (only for the log (y) 
case). We note that this is reminiscent of Crabill's exclusion principl^for the control of M/M/1 


^which states that for a M/M/1 queue with controllable service rates, the stationary optimal policy that minimizes 
the time average of the single stage cost Q{t) + l3ca{n{Q(t))) id > 0), uses only service rates which are such that 
Cs(s) = c(s), where c(.) is the lower convex envelope of the service cost function Cal.). 
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queues with non-convex service costs |25|, but optimality in our case is only in the asymptotic order 
sense. From this analysis, we can conclude that an asymptotic characterization of the minimum 
average queue length in the asymptotic regime 5? for admissible policies, can be obtained from the 
techniques in Chapter 4 by considering the lower convex envelope of the service cost function, even 
if the service cost function is not convex. 
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Appendices 


6.A Optimization problem (6.11) 


Consider the optimization problem (6.11) : 

... 1 


minimize --E 

TT AAC 


S'min {^^e-^<^^o{p,Q)+pS\n\M\ 
such that EjrS > \Nc + e, 


(6.25) 


where vr is any distribution for S and e > 0. We denote the optimal value of the above problem by 
Pe{()- For any distribution vr, we have that the point 


EttS, 


1 


XN, 


■E, 


Smm('l,e-'^^®o(^’2)+pSln|A4| 


lies in the convex hull of the set of points 

1 


XNr_ 


s mm 


^-N^Eo{p,Q)+ps\n\M\ 


I s G 


With the constraint E^^S > XNc + e, it is clear that the there would exist some distribution vr' 
such that the point (Et^/S, ^^E^/ [Smin (l,), with Ej^/S = XN^ + e, lies 

,(s,N 

XNc 

is optimal, and we have that 


on the lower convex envelope of (the curve AC, as shown in Figure 6.3(a)). Therefore, vr' 


XN0^-NcEoip,Q)+p(XNc+e)ln\M\ jf g < (XN^ + c) < s', 

^((AA^c + e) — s') + otherwise, 


(6.26) 


where s' is such that 




,-NcE(^{p,Q)+ps ln|Air 


ds 


|s=s' — 1- 



(a) Illustration of the convex hull (the region (b) Illustration of the (<5, e)-optimal distri- 
between and including the curves AB and AC) bution : (i) for ei, the distribution has 
and the optimal values Pe(ei) and Pc(^ 2 )- Pr {S = XNc + ei} = 1, and (ii) for £ 2 , the 

distribution puts mass at s' and si. 
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Now we obtain distributions which are (5, e)-optimal, where <5 > 0. A distribution d is said to be 
(5, e)-optimal for (6.25) if 


1 


Ed 


Ed5 


\Nc + e, and, 

Pe{e) + S. 


We note that if : (i) XNc + e < s' then the distribution that gives probability 1 to XNc + e is (<5, e)- 
optimal V(f > 0, (ii) AA'c + e > s' and 6 > 5h = s' — then a distribution that 

gives probability one to XNc + e is (5-optimal, and (iii) XNc + e > s' and <5 < 6b, then a distribution 
that gives mass to two points s' and si, characterized in the following lemma, is (5, e)-optimal. 

Lemma 6.A.I. If XNc+e > s' and 6 < 6b, then the distribution d defined as follows is (<5, e)-optimal. 

Let si = ^ [(AiVc + e - s') + -5] - ^^(^'N+(XN,+e-s’) _ 


Pr{S = s'} = 
Pr{S = si} 


Si — (AA^c + e) 
Si — s' ’ 
_ XNc + e-s' 
Si — s' 


We provide an outline of the proof. We note that if the distribution needs to achieve only Pe(e) + 


6, then we can choose the points pi = (si, Cs(si, A'c)) ( 3 S shown in Figure 6.3(b)) and p 2 = 
(s', Cs(s', A'c)). The point (XNc + e,Pe(e) + 6) lies on the line joining these two points and is 
therefore a convex combination of the points pi and p 2 - Solving for si and the convex combination 
leads to the proof of the above lemma. 

We note that the (<5, e)-optimal distribution has finite support for any <5 > 0 and e > 0. 


6.B 


Proof of Proposition 


or 


Proof. For the policy e, under stationary conditions, we have that Q = s{Q) and t(s) = [as + 6 ] , Vs 
Suppose Ett^Q < oo for e, then we have that '^■weQ = X¥,t^^t{Q). For brevity, in this proof we 
use E [.] to denote Ett^ [.]. Since as + 6 < t(s) < as + 6 + 1, we obtain that EQ > and 
EQ < - 33 ^■ Also < Er(Q) < We also have that 


Eg2 


EQ^ 


= a^ET{Q) + X‘^ET{Qf, 

< a^iaEQ + b + 1) + {a^EQ^ + {b + if + 2a{b + 1)EQ) or, 
(6 + 1 )ct2 + {b+ 1)2a2 + + 2a{b + 1)A2) 

- (1 - a2A2) • 


For the policy e, EQ < 


EQ{aQ+b+l)+E^2P+MsP±l±lP 

MQ) 


. Substituting the upper bounds for EQ‘^, and 


EQ, and the lower bound for Er(Q) in the above expression leads to the upper bound on the 
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average queue length for the EXH policy. 


□ 


6.C Proof of Lemma 16.4.5 


Proof. We note that = inf {q : ES'(g) > s' — e}, for e such that 0 < e < s' — A, for A in RA2. 
We note that by definition, Vg > g<j/, 


IE5'(g) > s' — e, 
S{q).dP{S{q)) > s'-e. 


(6.27) 


Then, 


rSmax 

/ AdP(5(g))+ / Sma.dPS{q) > s'-e, 

70 Ja 

/\{l - Pr {S{q) >/\}) + SmaxPr {S{q) > Ia} > s'- e, or, 

Pr{5(g)>A} > 

^m.ax ^ 


Thus for any g > g^/, Pr{S'(g) > A} > (5^ > 0, where Sg = 


s'-t-A 

Smax ^ 


□ 


6.D Upper bound for average queue length for the policy /i in Propo¬ 
sition 16.3.41 

For the policy // with parameters Nc and r, we have that a batch of size s(g) = min(g, rWc) is 
served when the queue length is g. We note that if the arrival distribution is such that Amax < r, 
so that 74[1] < rNc, then the average queue length = XNc- 

The following upper bound, holds for any arrival distribution with = E (A.[l]^) < oo. From 
( |6.lD we have that 

E {Q[m + 1]'^ - Q[nif\Q[ni] = q] = -2q{s{q) - XNc) + siq)"^ - 2s{q)XNc, 

= -2qNc{r - A) + 2q{rNc - s(g)) + EA'^ + s{q)'^ - 2s{q)XNc. 

Now, by taking expectations with respect to the stationary distribution vr of /x, we have that 

E^Q = [2E^ [Q{rNc - s(Q))] + E^^ + e^s(Q)2 - 2X^N^] , 

= [(t^Nc + 2E^ [Q{rNc - s(Q))] + E^s(Q)2 - X^N^] . (6.28) 

We now simplify and provide an upper bound for 2 E 7 r [Q{rNc — s((5))] +E^s(Q)^ — X'^N^. 
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We denote the stationary probability Pr {s(Q) = rNd = Pr {Q > rNc} by pr- Then we have that 
2E, [Q{rN, - s(Q))] + E^s(Q)2 - 

= 2 / q{rNc-q)dTT{q)+ f q‘^dTT{q) + prV^N^ - X^N^, 

J q<rNc Jq<rNc 

= 2 / qrNcd7r{q) - j q^d'K{q) + PrV^ 

J q<rNc J q<rNc 

= 2rNc [AiVc - PrrNc] - f q‘^d7r{q) + Prv'^N^ - 

J q<rNc 

= 2rXNl - [ q^dn{q) - p^N^ - 
J q<rNc 

= XN^{r — A) + rXN^ — prr'^N^ — f q‘^dTr{q). (6-29) 

J q<rNc 

Therefore, 


2E^ [Q{rNc - s{Q))] + E^s(g)" - < XN'^{r - A) + rXN^ 


(6.30) 


Substituting in (6.28), we obtain that 


Tn ^ ^ rXNc 

’^*'5 - 2(7^ ^ ~ ^ 2(7^' 


We now present another upper bound on 2E^ [QirNc — s(Q))] + E7rs(Q)^ — X'^N^, obtained by 


lower bounding pr in (6.29). The lower bound on pr is obtained under the assumption that 
Pr {A[l] > rNc] > 0. Let pi = 1 — pr- We note that pi is the fraction of time a batch size 
less than rNc is used. We note that the evolution of {Q[m]) can be divided into cycles of random 
duration. Each cycle comprises of two periods, where each period is also of random duration. The 
first period starts in slot m, if Q[m — 1] > rNc and Q[m] < rNc- The duration of the first 
period is distributed according to a Geometric distribution with mean pc{A[i]>rN } ' Following the 
first period, we have the second period which starts in a slot m such that Q[m — 1] < rNc and 
Q[m] > rNc- We note that throughout the second period the queue length is greater than or equal 
to rNc and a batch service of rNc occurs in each slot. 


Let Qp be the random queue length at the start of the second period. We note that Qp ~ A[l] 
conditioned on {^[1] > rNc}. Let Qp = qp. Since {Q[m]) is Markov, the duration of the second 
period Tp{qp) is then 

Tp{qp) = min [t\qp+'^ [A[m] - rNc] < rNc 

( m=l 

Then, applying Wald’s lemma for qp, and taking expectations over the distribution of Qp, we have 



256 








that 


We note that 


ETp{Qp) > 


IE[yl[l]|A[l]>rAfc] [Qp - rNc] 


rNr, - \Nr, 


Pi = 


< 


Pr{yl[l]>rArc} 


Pr{A[l]>rAfc} + 


Pr{A[l]>rNc} 

1 I lE[>l[ll|A[l]>riVc] 

Pr{A[l]>rN^} rNc-XNc 


C in' /a>riVc U 4 -U 4 - 

Since IE[A[i]|A[l]>riV,]Qp = Pr{l[l]>rN,} ’ 


Therefore, 


Pi < 


1 


1 + 




rN^-\N^ 

Ncir - A) 


Pr > 


Nc{r - A) + - rNc)dPA[i](a) 


fa>rNj^ - ^^c}dFA[l](a) 


Nc(r - A) + - r7Vc)d^A[i](«) 


Substituting the above lower bound in the RHS of (6.29) and using q^d'K{q) > 0, 

that 2AiV2(r - A) + (a - 25 ^) 


^a>rNS^-^^c)dPA[l]{a) 


< 2XN^^{r - A) + | A"-^ 

Nc{r - A) + - rNc)dPA[i]{a) 

The above expression can then be simplified to 


2XNl{r - X) + {r - X)N[ 


(A^iVc - 

Ia>rNS(^ - ^^c)dPA[l]ia) 

(r + A)) 

Ncir- 

A) + 

/a>riV,(« - rNc)dPA[i]ia) 



Therefore, 2E^ [Q{rNc - s(Q))] + E^s(Q)^ - X^N[ 

< 2XNI{t - A) + (r - X)Nl 


(A^iVc - 

Ia>rNS^ - ^^c)dPAll]ia) 

(r + A)) 

Ncir - 

A) + 

Ia>rNM - ^^^)^PA[l]ia) 



we have 


(6.31) 


(6.32) 
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Figure 6.3: Illustration of Cs(s, Nc) and its lower convex envelope, for Nc G {100,150, 200}, for the param¬ 
eters in Example 1 


Substituting in (6.28), we obtain that 




(A^iVc - 

Ia>rNS^ - ^^c)dPAll]{a) 

(r +A)) 

Ncir- 

A) + 

Ia>rNS^ - ^^c)dPAll]{a) 



6.E Examples for Cs{s,Nc) in Remark 6.4.10 


In this section we provide representative numerical examples which illustrate that the lower convex 
envelope c(s) of Cs{s,Nc) has a similar form as c{s). The transition probability matrices and the 
input distribution Q have been generated randomly for these examples. 


Example 1 : For this example we take \M\ 
transition probability matrix Py\x : 


0.0459 

0.2101 

0.1339 

0.1138 

0.1094 

0.0108 

0.2152 

0.0646 

0.0593 

0.0210 

0.1251 

0.1038 

0.0403 

0.1645 

0.1146 

0.0728 

0.1252 

0.1291 

0.1038 

0.0062 

0.1000 

0.1678 

0.0060 

0.1042 

0.0149 


2, and a DMC with \X\ = 5 and |T| = 10. The 


0.1212 

0.0348 

0.1859 

0.0309 

0.0140 

0.0341 

0.2328 

0.1601 

0.0612 

0.1409 

0.1375 

0.0872 

0.0727 

0.0657 

0.0886 

0.1191 

0.1417 

0.1202 

0.1211 

0.0607 

0.1505 

0.1352 

0.0174 

0.1673 

0.1368 


The input distribution Q is (0.2018,0.2551, 0.2515,0.0943, 0.1972). 


Cs{s,Nc) for three values of Nc is shown in Figure 6.3 


Then the numerically obtained 
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Figure 6.4: Illustration of Cs(s, Nc) and its lower convex envelope, for Nc G {100,150, 200}, for the param¬ 
eters in Example 2 

Example 2 : For this example we take \M\ = 20, and a DMC with \X\ = 5 and |3^| = 5. The 
transition probability matrix Py\x is : 


0.0649 

0.2915 

0.1642 

0.1986 

0.2808 

0.1557 

0.2722 

0.0509 

0.1063 

0.4148 

0.1883 

0.1538 

0.2409 

0.1187 

0.2984 

0.4160 

0.0809 

0.0959 

0.2615 

0.1457 

0.0860 

0.2355 

0.2115 

0.1459 

0.3212 


The input distribution Q is (0.0739,0.0141, 0.2423,0.3808, 0.2889). Then the numerically obtained 


Cs{s,Nc) for three values of Nc is shown in Figure 6.4 We note that in all these cases the form of 


the lower convex envelope of Cs{s,Nc) is the same as that illustrated in Figure 6.2 
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CHAPTER 7 


Conclusions and Scope for Future Work 


We first summarize the main results which are obtained in this thesis. Motivated by the already 
available results on the monotonicity property of any stationary deterministic optimal policy for the 
unconstrained tradeoff problem, in this thesis we consider the constrained tradeoff problem for the 
set of admissible policies, which are defined to be monotone. 

In Chapters 2 and 3, for the state dependent M/M/1 model, using geometric bounds on the sta¬ 
tionary probability distribution of the queue length for admissible policies, we obtain the asymptotic 
behaviour of the solution to the constrained tradeoff problem for admissible policies in the regime 
3^, as well as asymptotic bounds on any sequence of order optimal admissible policies. We identify 
a case in which the average queue length grows only to a finite value in the regime K. For the cases 
for which the average queue length grows without bound, we show that the asymptotic behaviour of 
the average queue length is either 0 (log(y)), 0 (y), or 0 The asymptotic behaviour of 

the average queue length is determined by: (i) the nature of the service cost function, i.e., whether 
it is piecewise linear, a corner point, or strictly convex, at the value of A or u~^{uc), and (ii) the 
extent of freedom that we have in the control of the arrival rates (A(g)) and service rates 

Guided by the analysis in Chapters 2 and 3, we obtain asymptotic bounds for the solution of the 
constrained tradeoff problem for a discrete time model with a fixed environment state in Chapter 
4. We again consider admissible policies, for which geometric bounds on the stationary probability 
distribution for the queue length are obtained. The forms of these geometric bounds are motivated 
by the geometric bounds which were derived for state dependent M/M/1 model in Chapters 2 
and 3. Using these geometric bounds, asymptotic lower bounds are derived for the solution to the 
constrained tradeoff problem in the regime 3?. Asymptotic upper bounds are also derived, using 
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which a complete asymptotic characterization of the tradeoff in the regime K is obtained in two 
cases. Asymptotic bounds on any sequence of order-optimal admissible policies are also derived. 

In Chapter 5, we first consider the tradeoff of average power and average delay for a noisy point-to- 
point link with fast fading. Asymptotic bounds for the solution to the constrained tradeoff problem, 
for admissible policies, in the regime K are then derived by extending the results of Chapter 4. 
We also compare the asymptotic lower bounds which are obtained from a real valued approximate 
queueing model to those from the original integer valued queueing model. We find that the real 
valued approximate queueing model with a strictly convex cost function underestimates the average 
service cost and average delay for the original model. We show that a more appropriate approximate 
real valued queueing model is one in which the service cost function is the piecewise linear lower 
convex envelope of the service cost function for the original model. We also obtain asymptotic 
lower bounds for the constrained tradeoff problem for: (i) a model with admission control, and (ii) 
a single hop network model. Asymptotic lower bounds are also obtained for the case when the 
arrival process and fading process are ergodic. 

In Chapter 6, we consider the tradeoff of average error rate and average delay for a noisy point- 
to-point link. We obtain the exponential decay rate of average error rate with respect to average 
delay, in the regime of large average delay, for fixed length random block coding schemes, with 
control on the codeword length parameter Nc- Using results from Chapter 4, we then obtain an 
asymptotic characterization of the tradeoff of average error rate and average delay, for the set of 
admissible policies, for a fixed Nc- One of the main contributions in Chapter 6 is the analysis of 
the constrained tradeoff problem for a non-convex service cost function, unlike the convex service 
cost functions in the earlier chapters. We show that the asymptotic behaviour of the tradeoff in 
the regime K is determined by the lower convex envelope for any non-convex service cost function. 


7.1 Scope for future work 


We note that all the asymptotic bounds obtained in this thesis are order bounds. From numerical 
computation, we have observed that the constants which are involved in the upper and lower bounds 
are weak. Tight non-asymptotic bounds for the constrained tradeoff problem are still not available. 

We now discuss some specific problems that arise from the analysis in the previous chapters, for 
which some initial results have been obtained. 
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7.1.1 Order optimality of policies obtained from a fluid model 


A simpler deterministic model for the evolution of the queue length, for the state dependent M/M/1 
model in Chapter 2, is the trajectory q{t) obtained from the fluid model: 

= A -/x(f),g(0) = go, (7.1) 

where /i(f) is the service rate at time t. The fluid model can be interpreted as a limiting form 
of the evolution of the integer valued queue length, when both time as well as queue length are 
scaled. Such fluid models can also be obtained for the discrete time model in Chapter 4, as well 
as for general networks, e.g. Chapter 10]. In Chapter 10] and |[T7|, it is shown that the 
policy /i(t) which minimizes the total cost J{qo) for the deterministic fluid model, can be used to 
obtain good policies for the unconstrained MDP for the original stochastic state dependent M/M/1 
model. Our objective is to investigate the order optimality of policies which are obtained from fluid 
models in the asymptotic regime SR. 


The state transition diagram for the unconstrained MDP is the same as that in Figure 2.1 with the 
action at each state q being the service rate g,{q) G {0, g,i, • • • , fix}- We also note that the optimal 
policy for the MDP with single stage cost q + /3c{fi) is the same as that for the MDP with single 
stage cost q + — c(A)). We construct a heuristic policy for the model, when c(/i) = 

for ^ G 5 = {0, ^ 1 ,..., /iif}, based upon the analysis in Chen et al. [T7j. As in Chen et al. jl7j . 
we consider the fluid model with a modified single stage cost q + jd [c(/i) — c(A)]'*^ and the 

service rate at time t, n{t) G M+. Then 


J(go) = inf / {q{t) + /? [c(/i(t)) - c(A)] + } dt. 

mW Jo 


If c(/i) = fj?, then from |T7j we have that J{q) = |g 2 ^. Furthermore the optimal service rate 

function for which 

poo 

J{qo)= / {q{t) + /d[cin*{t))-c{X)]'^}dt, 

Jo 

is given by the state dependent policy fJ.*{q) = (g//3)2 + c(A). Motivated by this development, we 
define the fluid policy 7 f ,/3 for our M/M/1 model as choosing the service rate 


dF,i3{q) 



where [x]^ means that we pick a service rate in S which is closest to the argument x. Interestingly, 
the policy 'yF,j 3 does not depend on A but varies only as a function of /3. We note that is 
admissible for any A < /jki since there exists a queue length q such that t'^F,i3{q) = d-K and from 
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the definition, fj,F^p{q) is non-decreasing in q. We illustrate the order optimality of 'yF,i 3 only for 
case 2. We first obtain an upper bound on QijF,^) using Proposition 2.A.1 
for e > 0, if there exists a such that ^F,p{(le) — A > e, then Q( 7 f,/ 3 ) < 


It can be shown that 


<?e(e + A) I \+^lK 


We note that if A < for case 2, we can pick e = — A. Then qe < P 

we obtain that Q{'yF,i3) = C>(/3). 

We have that C{^F,p) 


+ 

2 


2 e 

2 


-|- 1. Thus 


ku 

k<.ki k—ki 

< c{fiki-l)Pr {^J.F,|3iQ) < + c{fj,K)Pr {^J-F,|3{Q) > ^J-u} 

fell 

-l-c(A)-f ^ 7r^(A:)m(A - ^fc), (7.2) 

k=ki 

where m is the slope of the line joining (/x;, c(/x;)) and (//„, c{^u))- We note that since 7 ^?^^ is admis¬ 
sible, Ef=o = A. Hence, we have that Y.k<ki (pk - A) -b Efc>fc„ (Afc “ A) = 

(■^ “ Afc)- Since for k < k, fik < A, we have that J2k=ki “ Afc) < (AiC - A) Pr {^f,/ 3 (Q) > / 

We note that Pr{fiF,fs{Q) > An} = 7 r(g) and Pr{/iF,/ 3 (Q) < wl = 7 r(g). From 

the birth-death structure of ( 5 (t) under 7 f,/ 3 , we can show that 


Tr{q) < 7r(qk^ + 

g>Qku 



Pku+l 
Afcti-i-i “ 



g^Qki-i 


h- 






Fki-i 

^ A 


We note that 7 r(gfc„-b 1 ) = 7 r(gfc^_i) ^ and 7r(gfc,_i) = 7r(gfcJ T From 

the definition of /xf,/? we have that q^^ — qk^-i and qki — qki-i are both 0(/3), as /3 f co- Since 

'^('Zfcu-i) and F(gfc;_i) are both bounded above by one, we have that Pr {^f,i 3 {Q) > An} as well 
as Pr {^f,i 3 {Q) < A/} are both where 0 < A< 1 . Then we have that C{'yp) — c(A) = 

and Qi'jp) = 0{j5). In summary, we have shown a new optimality property of heuristic policies, 

obtained from a fluid model analysis. This raises the question whether such fluid policies are order 

optimal even for general network scenarios. 


7.1.2 Extensions to systems with service time control 

We note that the service rate control variable has been the service batch size for all the discrete 
time queueing models considered in this thesis. However, there are scenarios where both batch 
size and batch service time can be controlled, e.g., for a noisy point to point link which uses block 
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coding, both the number of encoded message symbols (the batch size) and the codeword length 
can be dynamically controlled (the service time for the batch) to tradeoff the average error rate 
with the average delay of the message symbols. Such models also arise as a special case, when 
single decision policies with observed initial information are used, for the general class of renewal 
models considered by Neely |46j. We note that for such models the service cost is then modelled 
as a function of both batch size and batch service time. 

We now comment on how asymptotic lower bounds can be derived for such models, using the 
methods in Chapter 4. We note that for stationary policies, that decide on the batch size and batch 
service time as a function of the current queue length, the queue length evolution sampled at the 
decision epochs evolves as a semi-Markov process. By uniformization [75] the average cost and 
average queue length for a semi-Markov process can be obtained via an equivalent Markov process. 
The stagewise drift of this equivalent Markov process depends on both the batch size as well as 
the batch service time. We expect that the analysis of this equivalent Markov process is similar to 
the analysis carried out in Chapter 4, but with the above drift function, under the restriction to 
admissible policies. 

We consider a simplified model in [52], wherein asymptotic lower bounds are derived for a continuous 
time queueing model with Poisson arrivals and service time control with service batch size fixed to 
be 1. For this model, as the service batch size is fixed to be 1, whenever there is service, the service 
cost is a function only of the batch service time, which is real valued. For the above continuous 
time model, where the queue evolution is on the integers, if the service cost per unit batch service 
time is a strictly convex function of the batch service time, then we obtain that the minimum 
average queue length grows as when the average service cost constraint is V more than 

the minimum average service cost required for stability. Obtaining asymptotic bounds for queueing 
models where both batch size and batch service time can be controlled, is a problem which has 
scope for future work. 
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